Incorrect Predictions

I've been studying the realm of LLMs for a while and just wanted to throw some thoughts out to start a conversation on the topic. I work with large language models (and, to a lesser extent, vision, TTS, and STT models) every day these days, both in and out of work in multiple ways, including personal entertainment, personal creative projects, and personal practical projects. Coding, writing, research, and analysis; I use these tools everywhere. These tools are only going to get better. The hardware required to run them will only get more powerful and cheaper to manufacture. We're at a place where the current state of the art frontier models are already capable of performing feats in hours what would have taken teams of people months. And while it's not perfect in the slightest and requires deep experience and careful curating to utilize well, it will get easier. The big things coming down the pike in AI aren't on most people's radar. For example, the use of GPUs for training and inference is largely an accident of availability. GPUs are the best of a lot of poor instruments to use to perform the calculations required for LLM inference. That's changing, and dedicated hardware is being developed in a variety of different ways that will outpace current GPUs enormously in power and cost and capability. The transformer model - the underlying architecture to all large language models - is not the end of the research line into large language models. There are companies and teams working on what comes after transformers, aiming for continuous learning models that have the same capabilities but are not restricted to their pre-training and can instead learn on the fly. There are also models that incorporate Bayesian statistical subunits capable of gauging certainty and surprise, both useful indicators that the model might not be confident in what it's about to do or say. There are also efforts to make models far more transparent so that how a conclusion was arrived at or whether the model was about to misbehave just to get ahead will be observable. All of this to say: one of the most difficult things to do in this tumultuous, quickly-changing time is gauge what next year let alone the next five years is going to look like. I have high hopes, however.