It's 15th September 2024, 3 days from release of gpt-o series models and things are going to change over next 2 years.
The latest gpt-o series model are a new paradigm - of reasoner models. They perform much better at specialized tasks like STEM reasoning, writing code etc. This is just the beginning and it's only about to get better.
OpenAI engineers are very confident about scaling (and optimizing these new series of models).
It's important to emphasize that this is a huge leap /and/ we're still at the start
— David Dohan (@dmdohan) September 12, 2024
Give o1-preview a try, we think you'll like it.
And in a month, give o1 a try and see all the ways it has improved in such a short time
And expect that to keep happening
People have already used o1-mini model to perform at a master-level rating performance on codeforces contests. One should note that codeforces problems lean on the novel side so these models are able to reason pretty well.
Another tweet showed that o1-preview model is SOTA at a medical benchmark.
--
This is just the progress from the o1 series of models. OpenAI mentioned they will continue to release models under the gpt-series too. We will also have an Opus 3.5 release and Gemini 2.0 release from which people have high expectations. Magic dev is also cooking some cool stuff that Nat Friedman is bullish on.
Below this line what i say can sound delusional or overthinking
Bottomline, I feel things are going to dramatically change over the next 1.5-2 years and there is a need to adapt quickly. Software engineering as we know today is gonna change in next few years. People who are living in their own bubble and not paying attention to the AI advancements are in for a shock.
I don't have much answers, mostly questions. How should one adapt to the changes? What should one focus to upskill in order to keep their jobs. Jobs are not going to get replaced but the roles are gonna reduce across fields - for entry to mid level roles. What are some things one should not focus as much now.
How can we use LLMs to improve our own understanding of things? This reminds me I need to watch Terence Tao's video on how they used LLMs to write that particular proof.
I feel working on AI and around AI is a good strategy to keep your employment. A lot of capital is going into this sector and demand is only gonna increase.
Go deeper into your stacks (for ex: learn about the inference stack). Learn to abstract think well. Keep tinkering with new models as they come - atleast the SOTA ones like Sonnet 3.5 and o1-mini. Don't be a purist who says no I will not utilise LLMs to code. Take their help or you will stay behind. With the o1 release, you can't say that LLMs don't help in writing code now. Also, if Sonnet 3.5 was good enough for Karpathy, it should be good enough for you too. Put your ego aside and take help.
I am thinking to spend more time learning about distributed systems too - reading DDIA as of now apart from learning about the GenAI stuff. I am also looking for GenAI roles so that's why too. Anyways, scalability of systems is gonna be lindy.