Don’t inference providers also provide APIs for cost checking? I imagine you’d need to have a cost-checking service that cancels any CI jobs running an agent. If possible, I’d chain the CI jobs so that one job creates a token with metadata and triggers a new job which runs the agent. The cost-checking service could then trackspending and call the CI API with a kill command for the job whose API token has crossed the limit.

Maybe the inference platforms even allow calling webhooks on certain events. You would still need to identify which CI job to kill, hence my suggestion to create tokens per job, that would allow you to target the job using that token.

Alternatively, you have a second process running in the CI job that pings the inference providers API and kills the agent when a certain condition is met.

100%

I got this my superior who decided AI allowed them to code fixes and we were stuck with buggy code that wasn’t merged, because the superior had moved on to a more important task requiring more AI “support” and couldn’t process the comments. Things could’ve been faster had we implemented things ourselves.

When you aren’t out in the fields dealing with stuff, you don’t know what’s important to the workers. “Jumping in” for a minute to do a task might actually slow down the team if understanding of day to day struggles isn’t present.

Is there good jj integration in editors?