I tried getting it to generate a Red-Black tree in Java but it cuts off half way through.
I suppose you could divide and conquer with smaller parts of the algorithm, but then we'd need a "meta AI" that can keep track of all those parts and integrate them into a whole. I'm sure it's possible, don't know if it's available as a solution yet.
Think of it in terms of limited workspace memory. How much of a program can you really fit in your head at once?
Both less and more than GPT, because humans can learn from limited input and also we have a lot of tricks for escaping our horribly limited context size. GPT probably has a larger context than humans, but it’s worse at everything else—to the degree that’s comparable.
I wouldn’t bet on that changing soon. I also wouldn’t be on it staying the same.
I suppose you could divide and conquer with smaller parts of the algorithm, but then we'd need a "meta AI" that can keep track of all those parts and integrate them into a whole. I'm sure it's possible, don't know if it's available as a solution yet.