• NewNewAugustEast@lemmy.zip
    link
    fedilink
    English
    arrow-up
    1
    ·
    14 hours ago

    A lot of that was scraping. A huge amount. Just take everything possible. Hopefully that will slow down, or as I did, I allow requests and reponses, but scraping gets kicked.

    But it is getting hard and harder to self host. Large websites and hosts (gitlab, self labs) get slammed and even DOS. Its been rough.

    There are huge network upgrades in the works to try and deal with this 3.2T are being looked at already. But of course electricity is going to hit a hard ceiling both for the data center and the network center. Hence the costs you are talking about.

    I am more concerned with the fact that if everyone is asking AI, who is going to put the answers somewhere to get them? Who is going to self host and publish like I do today? It already is rare to see people running their own websites anymore. Lemmy is pretty amazing for that, but it too has gotta be having a huge amount of traffic at each instance.

    I guess I am saying I am aware of your concerns and I have mine too.

    • Jiral@lemmy.world
      link
      fedilink
      English
      arrow-up
      1
      ·
      13 hours ago

      It feels a bit like in Cyberpunk, where the wild Internet has become unusable or even dangerous and only big corpo controlled parts are still accessible to the masses.

      The other topic you mentioned, LLM’s are going to hit a wall and they’ll do so hard. Training will be increasingly relegated to information predating large scale use if LLMs to avoid rapid degradation of the models by training it with LLM output. That in itself creates a huge problem the more time passes. Nothing is safe btw, analog media are increasingly just as tainted.