There Will Be Inference

Coding agents are one of the biggest unlocks in this AI era, and after establishing that I can in fact build a pretty decent web app, I decided to challenge myself to make it a mobile app and get it on the app store.

Quickly I learned that if you want to put something on the app store, the mobile app needs to feel meaningfully different from the web app, and the baseline recommendation is two distinct features that are mobile only. So the good news is I always knew I wanted to add push notifications, so that was one. And I also wanted to try adding an AI feature, so I'd make that one mobile only, and boom, two.

Recently I decided that I should understand how to actually serve AI models in production, and I've worked at an AI startup too long to not know how that works. So I decided to ask my *insert favorite LLM chat of the moment* to give me a lesson on inference, and quiz me along the way.

It was great to have a general picture for how this all works, but I didn't really understand what that meant in practice.

Enter my web app — soon to be mobile app — veggie.sh. A photo recognition feature that could recognize all the plant-based foods on your plate and add them to your daily log seemed like a natural fit, but again I was kinda like: ok well where do I start.

From my little lesson, I knew this was the moment inference went from a vague idea to a concrete function in my app. I also knew that I'd need a provider that hosted models to handle all the infra that I couldn't build myself. So I just kinda picked an inference provider that was top of mind, which happened to be Together AI.

What was even better was that Together had a search feature for the models they hosted where I could filter by the input and output I wanted and then I just picked one — and figured I could evolve it over time if needed. I ended up going with Gemma 4 31B and it's working great so far.

As I mentioned, push notifications were also on my mind. And that's when I got to understand the wonderful world of the Apple app store, because it turns out you can't test push notifications without making an Apple developer account. Which really got me wondering — I know that Apple does not have much incentive to make the app store more accessible, but bruhh I was done dirty at every step.

But I'm proud to say that my app is officially in TestFlight, and I will try to get it on the store sometime this weekend!

The only thing I don't fully understand is when I have to resubmit the app to the store for review, and where I can test new features before pushing them to prod. However, what I've learned so far is that you just take it step by step, ask an LLM when you don't know, curse at said LLM when it gives you a dumb answer, and eventually you'll figure it out!