• perestroika@slrpnk.net
    link
    fedilink
    arrow-up
    4
    ·
    edit-2
    1 month ago

    Regarding AI, some simple examples of likely computing expense:

    • performing 1 + 1 in assembly language: 1 CPU cycle
    • performing 1 + 1 in a high level interpreted language: maybe 10 CPU cycles
    • performing 1 + 1 using an LLM: millions of CPU cycles
    • asking “why was the rabbit an animal after 1 + 1”: probably priceless, maybe a billion cycles
    • keepthepace@slrpnk.net
      link
      fedilink
      arrow-up
      4
      ·
      1 month ago

      Nowadays with JIT #1 and #2 are very close. There was less than a x2 performance gap when I compared a direct simple task in C and javascript.

      About #3 no one does that.

      #3 and #4 will be on the same scale, every token generated is roughly the same amount of CPU.

    • MysteriousSophon21@lemmy.world
      link
      fedilink
      arrow-up
      2
      ·
      8 days ago

      And that translates directly to energy consumption - a single ChatGPT-style conversation can use as much electricity as charging a smartphone, which is why companies are scrambling to build more efficient AI chips and why they need so many power stations (you can compare some on gearscouts.com to see the best $/Wh values for emergency backup).