I asked Astra to try and fix an issue in my codebase I've been battling for a month. Day and half later on high and it's still circling the drain. Seems like with each new model we praise it based on toys, but then reality hits the ground soon and we wait for the next model. In the meantime, good ol' brain.

Have you tried turning it on and off? More seriously try to remove agent memory or make it refactor it or go through the docs and look for inconsistencies.

But I still don't know how people work with codex when it keeps resetting the context so often, I mean it's surprisingly good at making notes to itself and can follow a long task, but if there are multiple instructions it sometimes forgets some of them.

I just did that and turned on max, let's see if that thing is more of a force!

I'm curious how it went.

26% weekly limit left, 21+ hours strong and still going. I can see the progress though, but I am not confident it will finish before weekly limit is done for (ChatGPT Pro 20x subscription). Since I don't really use LLMs for anything else, except pestering gemini for stupid questions instead of straight up googling it, that's ok. I just wish it would either finish or let me hand it over to it once again whence limit resets.

I’m just trying to understand, there’s an issue you’ve been unable to deal with for a month, and Astra not being able to handle it in 36 hours is disqualifying?

Long time ago I had one particular hardware issue, which Opus 4.6 found a workaround to fix. I don't remember the workaround and being careless (I thought I could ask an LLM again if I needed) I lost that solution. Some time passed and I needed it again - neither one of the newer models is able to come up with a nice solution I had back then. They can solve the issue and find a different workaround eventually, but not as nice. On a side note, I could use 4.6 again of course and try to reproduce it, somehow I only thought about it now writing this comment.

Anyway, my point is that I think these frontier companies are advertising their one-shot model abilities, but underneath the models aren't getting so much better as they try to make you believe.

On and off for a month, not straight month and not dedicated time. I am quite confident I will be able to solve it within a week or two of dedicated time. Here, on the other hand, we have this supposedly powerful thing that can do magic, unlike myself. We'll see. I'm not even confident, by looking at what it does, that it will even be able to solve it. Only thing I saw that I like is it's better in communication style compared to Fable and Opus.

Can you say anymore about the issue it can’t solve?

Yeah absolutely. I've been working on my rendering engine on and off for the past 20 odd years. It has been used in commercially in my gigs. However, it's what's considered an offline renderer (like 30+ minutes to render a frame). Last few years, on and off again, I've been slowly working towards a limited in scope realtime version of it. One of the components I've managed to kind of bring almost to the finish line is for it to work with a limited VRAM budget and streaming geometry, where camera has a (I'm simplifying here) bubble around you (around, not a frustum in-front because I need semi global visibility since I'm doing realtime raytracing). This bubble as it moves dictates what geometry and resources are loaded, sometimes even ahead of time because I also do a form of motion prediction. Far out things, outside of the bubble, and things on the border of the bubble go through a coarse to fine representation. That way I can render distant stuff which I don't need much detail for it. This part is a bit buggy, to say the least, but also viably measurable so it's a prime candidate for an LLM to loop/goal over it. Or at least I though so. With max I can see now the progress, but with max I'm now 21+ hours into it and not a clear solution ahead yet. I think I'm running out of weekly quota on ChatGPT Pro 20x for this. I actually bought subscription exactly to drive this home since I don't use LLMs much anymore except gemini in lieu of googling.

I haven't told it how to do stuff, but what I want to do and it picked up my tests to see if it's making progress and I can see it's tuning the movement prediction and caching logic and it fixed few minor bugs. So, it seems to be doing the right approach.. it just takes an awful lot of time. Question is if it's a viable approach even though, but it didn't flag it as wrong. It actually agreed with me and unlike Claude I haven't gotten the impression it's much of a yesman.

Interesting, that was a bit hard to follow because of my own lack of ‘context’ :). But it sounds tough. I imagine since rendering has a visual component that adds to the difficulty for LLM. Though you said it’s measurable, is the measurableness visual? And it sounds like you just sent it off with no direction; but good prompting and direction can go a long way. Is the architecture and design and constraints well documented in the codebase?

It's both measurable 'headless' and visually. When I said no direction I mean that I didn't want to give it direction _how_ to do it but more what I want to do and how what I use to measure. Usually I found better results with LLMs by giving them the tools and describe the objective and let it figure it out rather than box it in completely. It did well on it, it's just.. well, it's taking its sweet-ass time :)

Heard. And Claude is even slower ><

As long as it keeps AI in the news

I made 3D sudoku, books on paper called SUDOCUBE, with a coded index for the clues, using Opus 4.7 I think it was, which came up with the algorithm and extended my own pre-LLM Kakuro codebase:

https://www.kakurokokoro.com/sudocube

A couple years ago I wrote a rough CLI based proof-of-concept of a different form of 3D Sudoku. Funny enough, just this morning I was thinking about using Claude to turn it into something actually playable.

Have you seen ColorDoku or MeowDoku? Much better for me to fill colors than numbers. Colors would fit perfectly in a 3D cube, just an idea

If it is only 4x4x2x2 then this has fewer hypersquares than a proper version of 4D tic tac toe, e.g.:

https://csclub.uwaterloo.ca/~r345liu/

In game design, restraint can be a virtue.

Yeah, if it was properly 4D it would be pretty unplayable I think, you’d need 4 each 4 layer cubes, and they could not be nested in any physical configuration.

You can show them as 4x4 boards of 4x4 squares.

Yes, but they can’t be meaningfully nested like the proposed one is. Thats the elegance of The solituon provided, but it necessarily sacrifices symmetry to get there.

Excellent job, this was very fun to play.

I liked the explanations, graphics, interactive tesseract, all nice attentions to detail.

How much guidance did you give Astra? Did you give a single prompt, an example site to mimic or was there more to the design and make process?

yes, a single prompt:

"Create a single html file 4d chess game. By 4d it means 4 dimenstional space. It needs to be playable with understandable rules."

It created the working game with the four boards but since it was impossible to understand from just the board I told it to add a Tesseract.

Not astra made but my mom and nephew say it's a fun thing to play https://chesspoly.com (https://apps.apple.com/gb/app/chesspolly/id6798789898)

I was expecting it more like a filled hypercube but its more like two surfaces? Probably better the way it is.

A quick reminder as to just how bad I am at classic chess, but this is really fun and brain bending

I think this (from 2000) describes exactly these rules: https://www.zillions-of-games.com/cgi-bin/zilligames/submiss...

There are many, many other such games with all sorts of variations on the rules.

that's a totally different game. It has 32 total pieces while this one has 20 and the piece configuration is also completely different.

I think flipping the tiling between light and dark on the top left and the bottom right (or vice versa) would make more sense. Then bishops stay on one color and the knight always alternates between colors (like 2d chess)

It feels unnatural that a white bishop can move to black squares.

Very cool! (For learning/ease of use, it would be helpful for me to be able tap enemy pieces to see where they could go next. (But with a different indicator.))

There is already 5d chess, it's been out for years on steam. It's very polished, has tutorials and puzzles. Time travel and parallel universes.

Did it also make a 4D chess Stockfish 19 for you?

Well it did make a very good search function that is almost like stockfish for humans in 4d chess

This is clearly not the same thing. Other than both of them adding additional dimensions to chess they have nothing in common.

that has 32 pieces this one has 20 not to mention that the peice setup and how the pieces travel across dimensions is totally different.

Do humans have to search all 10000 4d-chess ideas before LLMs to find the exact one that was plagiarized?

If you want to accuse it of plagiarism then yes (or have an LLM do it for you I guess)

very cool. that being said, I have never not wanted to play anything more than this :D