guess that explains why features like quantized KV caches are lagging behind
Yeah, I was using the TheTom fork for a while and not sure why TQ KV cache hasn’t merged yet.
Anti-AI sentiment is at an all-time high
I suppose the tech bros have understandably soured a lot of people on LLMs with all of the negative societal costs due to their greedy and irresponsible bejavior. On the other hand, the concepts of the perceptron and artificial neural networks from the 40s and 50s are finally coming to fruition and we have these things now that are legitimately artificially intelligent that we can run on our gaming PCs. They are overhyped and used in ways they make no sense, but they’re also useful as long as their limitations are kept in mind. From a technology perspective, they’re cool as hell.
Yeah, I was using the TheTom fork for a while and not sure why TQ KV cache hasn’t merged yet.
I suppose the tech bros have understandably soured a lot of people on LLMs with all of the negative societal costs due to their greedy and irresponsible bejavior. On the other hand, the concepts of the perceptron and artificial neural networks from the 40s and 50s are finally coming to fruition and we have these things now that are legitimately artificially intelligent that we can run on our gaming PCs. They are overhyped and used in ways they make no sense, but they’re also useful as long as their limitations are kept in mind. From a technology perspective, they’re cool as hell.