<?xml version="1.0" encoding="utf-8"?><feed xmlns="http://www.w3.org/2005/Atom" ><generator uri="https://jekyllrb.com/" version="3.10.0">Jekyll</generator><link href="https://blog.osoyalce.com/feed.xml" rel="self" type="application/atom+xml" /><link href="https://blog.osoyalce.com/" rel="alternate" type="text/html" /><updated>2026-10-11T18:10:32+00:00</updated><id>https://blog.osoyalce.com/feed.xml</id><title type="html">Blog</title><subtitle>A personal and professional blog.</subtitle><author><name>Nadim-Daniel Ghaznavi</name><email>nghaznavi@gmail.com</email></author><entry><title type="html">Animating Ax3l Playing Snake</title><link href="https://blog.osoyalce.com/journal/Animating-Ax3l/" rel="alternate" type="text/html" title="Animating Ax3l Playing Snake" /><published>2026-10-11T00:00:00+00:00</published><updated>2026-10-11T00:00:00+00:00</updated><id>https://blog.osoyalce.com/journal/Animating-Ax3l</id><content type="html" xml:base="https://blog.osoyalce.com/journal/Animating-Ax3l/"><![CDATA[<p>I’ve been tinkering with various incarnations of Patrick Loeber’s <a href="https://www.youtube.com/watch?v=L8ypSXwyBds&amp;t=4s">Train an AI to Play Snake</a> tutorial for a while now. It’s my relaxing goto project that’s fun, engaging and educational. While I’ve been watching hours of Snake Game simulations, the rest of you have only been looking at screenshots. Well, that’s all changed now:</p>

<p><img src="/images/2026-10-11-snake-game.gif" alt="Snake Game" /></p>

<p>Go to the <a href="https://snakeweb.osoyalce.com">Live Ax3l Site</a> to see more!</p>]]></content><author><name>Nadim-Daniel Ghaznavi</name><email>nghaznavi@gmail.com</email></author><category term="journal" /><summary type="html"><![CDATA[I’ve been tinkering with various incarnations of Patrick Loeber’s Train an AI to Play Snake tutorial for a while now. It’s my relaxing goto project that’s fun, engaging and educational. While I’ve been watching hours of Snake Game simulations, the rest of you have only been looking at screenshots. Well, that’s all changed now:]]></summary></entry><entry><title type="html">Dynamic DNS Service</title><link href="https://blog.osoyalce.com/journal/BMDynIP-Project/" rel="alternate" type="text/html" title="Dynamic DNS Service" /><published>2026-10-07T00:00:00+00:00</published><updated>2026-10-07T00:00:00+00:00</updated><id>https://blog.osoyalce.com/journal/BMDynIP-Project</id><content type="html" xml:base="https://blog.osoyalce.com/journal/BMDynIP-Project/"><![CDATA[<p>This project started as a cron job and a flatfile and -honestly- that would work perfectly well for a small home setup. Say, two or three machines. I’m currently running 8 machines which have various versions of different third party and custom software. Keeping my public IP’s DNS record mapped is just one more piece of my home lab infrastructure. So if you are wondering why I have an <em>enterprise-grade</em> solution, it’s because <em>that’s how I roll!</em> :D.</p>

<h2 id="bmdynip-overview">BMDynIP Overview</h2>

<p><img src="/images/2026-10-07-bmdynip-architecture.png" alt="BMDynIP Architecture" /></p>

<p><strong>BMDynIP</strong> is a dynamic DNS service for people who:</p>

<ul>
  <li>Have a dynamic public Internet IP address.</li>
  <li>Have a domain registered with GoDaddy, such as <strong>osoyalce.com</strong>.</li>
  <li>Want to host one or more Internet services.</li>
</ul>

<p>BMDynIP uses the <a href="https://github.com/godaddy/cli">GoDaddy CLI tool</a> to manage DNS A records for one or more hostnames, such as <code class="language-plaintext highlighter-rouge">www.osoyalce.com</code>. It keeps those records pointed at your current public IP address.</p>

<p>The service:</p>

<ul>
  <li>Runs as a Linux service.</li>
  <li>Provides a web interface for configuring the service and managing hostnames.</li>
  <li>Detects your public IP address and checks it periodically.</li>
  <li>Updates your GoDaddy DNS records when your IP address changes.</li>
  <li>Stores state and history in MariaDB.</li>
</ul>

<h2 id="bmdynip-web-interface">BMDynIP Web Interface</h2>

<p><img src="/images/2026-10-07-bmdynip-web-interface.png" alt="BMDynIP Web Interface" /></p>

<p>Here’a screenshot of the web interface.</p>

<h2 id="bmdynip-user-guides">BMDynIP User Guides</h2>

<ul>
  <li><a href="https://bmdynip.osoyalce.com/pages/architecture/">Architecture</a></li>
  <li><a href="https://bmdynip.osoyalce.com/pages/installation/">Installation</a>
    <ul>
      <li><a href="http://bmdynip.osoyalce.com/pages/root-authentication/">Root GoDaddy authentication</a></li>
    </ul>
  </li>
  <li><a href="https://bmdynip.osoyalce.com/pages/configuration/">Configuration</a></li>
  <li><a href="https://bmdynip.osoyalce.com/pages/uninstall/">Uninstall</a></li>
  <li><a href="https://bmdynip.osoyalce.com/pages/web-interface/">Web interface</a></li>
  <li><a href="https://bmdynip.osoyalce.com/pages/running/">Running and monitoring</a></li>
</ul>]]></content><author><name>Nadim-Daniel Ghaznavi</name><email>nghaznavi@gmail.com</email></author><category term="journal" /><category term="dynamic-dns" /><category term="dns" /><category term="godaddy" /><category term="self-hosted" /><category term="homelab" /><summary type="html"><![CDATA[This project started as a cron job and a flatfile and -honestly- that would work perfectly well for a small home setup. Say, two or three machines. I’m currently running 8 machines which have various versions of different third party and custom software. Keeping my public IP’s DNS record mapped is just one more piece of my home lab infrastructure. So if you are wondering why I have an enterprise-grade solution, it’s because that’s how I roll! :D.]]></summary></entry><entry><title type="html">Local GeoIP Database</title><link href="https://blog.osoyalce.com/journal/BMGeoIP-Project/" rel="alternate" type="text/html" title="Local GeoIP Database" /><published>2026-10-05T00:00:00+00:00</published><updated>2026-10-05T00:00:00+00:00</updated><id>https://blog.osoyalce.com/journal/BMGeoIP-Project</id><content type="html" xml:base="https://blog.osoyalce.com/journal/BMGeoIP-Project/"><![CDATA[<p>The <a href="https://bmgeoip.osoyalce.com">Bear &amp; Moose GeoIP Project</a> provides an easy-to-use GeoIP service for your applications and for interactive lookups.</p>

<p><img src="/images/2026-10-07-bmdynip-web-interface.png" alt="Web Interface" /></p>

<ul>
  <li>BmGeoIP has simple install.sh, upgrade.sh, and uninstall.sh scripts.</li>
  <li>Downloads public geolocation data from ipapi.is on a configurable schedule.</li>
  <li>Stores records in an optimized MariaDB database.</li>
  <li>Provides a web interface for interactive IP lookups at https://localhost:54300 and for configuring the download schedule.</li>
  <li>Accepts application lookup requests through ZeroMQ and HTTP.</li>
</ul>]]></content><author><name>Nadim-Daniel Ghaznavi</name><email>nghaznavi@gmail.com</email></author><category term="journal" /><category term="geoip" /><category term="ip-geolocation" /><category term="mariadb" /><category term="self-hosted" /><category term="homelab" /><summary type="html"><![CDATA[The Bear &amp; Moose GeoIP Project provides an easy-to-use GeoIP service for your applications and for interactive lookups.]]></summary></entry><entry><title type="html">An LLM to organize my Movies</title><link href="https://blog.osoyalce.com/journal/R3eling/" rel="alternate" type="text/html" title="An LLM to organize my Movies" /><published>2026-09-27T00:00:00+00:00</published><updated>2026-09-27T00:00:00+00:00</updated><id>https://blog.osoyalce.com/journal/R3eling</id><content type="html" xml:base="https://blog.osoyalce.com/journal/R3eling/"><![CDATA[<p>Our first TV was a small black-and-white thing where, on rare occasions, we’d watch <em>Animal Kingdom.</em> Then Beta and VHS came along, and you could watch them at home! Then came DVDs.</p>

<p>Things became digital. The cost of storage plummeted and so I started keeping digital copies all of the movies or TV shows that I liked. Why? Because I do rewatch things and I like collecting, and this seemed harmless.</p>

<p>And I thought, “Great! This will de-clutter my life, now all I have to do is organize a bunch of files!” I got rid of my DVDs and embraced the digital lifestyle.</p>

<hr />

<h3 id="the-swamp">The Swamp</h3>

<p>That was about 25 years ago. Now I have over a thousand movies, almost 200 TV shows, and seasons, and seasons, and seasons of episodes.</p>

<p>I tried to mostly keep things organized, renaming files and folders to keep things sane. I did a pretty decent job, but the sheer volume meant that over time finding or browsing was becoming more and more difficult, until it basically became impossible. There were just too many directories.</p>

<hr />

<h3 id="the-solution">The Solution</h3>

<p>The solution was obvious.</p>

<ul>
  <li>Create a browse and search interface.</li>
  <li>But first, clean the data so the code could find it.</li>
</ul>

<p>Creating a catalogue entry at the same time seemed natural.</p>

<p><a href="https://tmdb.org">The Movie Database</a> has a free, publicly available database of movie and TV titles, including poster art, episode descriptions, casts, and more. I could use that to create a really nice catalogue entry for my movies.</p>

<hr />

<h3 id="clean-the-data">Clean the Data</h3>

<p>There was <strong>absolutely</strong> no way I was going to do this by hand.</p>

<p>Writing software to handle all of the special cases was daunting. Tons of weird filenames and matching that to a <a href="https://tmdb.com">TMDB</a> entry would be super hard. Frankly, it was out of reach for me.</p>

<p>What I really needed was an intern, or a gopher (Go ‘fer <em>that</em>), or a robot to go through the directories one at a time.</p>

<p>I had just the thing! I’d recently downloaded <a href="https://huggingface.co/Qwen/Qwen3.5-4B">Qwen 3.5 4B</a> a small LLM, like ChatGPT, but running on my own computer. It was smart enough to handle the job.</p>

<hr />

<h3 id="llm-as-an-intern">LLM as an Intern</h3>

<p>I had just finished building <a href="https://ax3l.osoyalce.com">Ax3l</a>, an autonomous AI agent that was running thousands of simulations to tune a simple neural network. It turned a knob, ran a simulation, studied the results, turned a knob, ran a simulation…</p>

<p>This project was simple by comparison. I’d have the LLM:</p>

<ul>
  <li>Scan a directory.</li>
  <li>Match the contents to a <a href="https://tmdb.org">TMDB</a> catalogue entry.</li>
  <li>Rename and move the file to a new, tidy home.</li>
  <li>Create a catalogue entry in the database for the file.</li>
</ul>

<p>I named the project <strong>R3el</strong> and started designing.</p>

<p><img src="/images/2026-09-27-r3el-architecture.png" alt="R3el Architecture" /></p>

<p>Okay, not simple, but definitely something I could create using the <a href="https://ax3l.osoyalce.com">Ax3l</a> codebase as a starting point.</p>

<hr />

<h3 id="the-control-interface">The Control Interface</h3>

<p>I knew I’d eventually build a catalogue with a web interface to browse and search my collection, so I figured I might as well leverage that and build a control interface there.</p>

<p><img src="/images/2026-09-27-r3el-control.png" alt="Control Interface" /></p>

<hr />

<h3 id="the-event-log">The Event Log</h3>

<p>My experience working with LLMs taught me the value of fine-grained reporting. Again, I leaned on the <a href="https://ax3l.osoyalce.com">Ax3l</a> codebase and created a detailed <strong>Event Log</strong> so I could monitor, tune, and troubleshoot the process.</p>

<p><img src="/images/2026-09-27-r3el-eventlog.png" alt="Event Log" /></p>

<p>It took me most of one day to itemize and label all of the messages I was interested in, but it was an excellent investment and significantly helped focus my development efforts.</p>

<hr />

<h3 id="it-worked">It Worked!!</h3>

<p>It took some tuning; I added use cases, I improved the pattern matching, and I did a minimal amount of manual work sweeping through the filesystem. But it worked!!! For the next few days, I’d queue up a batch of directories and let it work.</p>

<p><img src="/images/2026-09-27-r3el-processing.png" alt="Runtime Screenshot" /></p>

<ul>
  <li>R3el scanned files</li>
  <li>It identified series and titles</li>
  <li>It renamed and sanitized the filenames</li>
  <li>It deleted the old empty directories</li>
  <li>It downloaded movie posters, cast and producer info</li>
  <li>And R3el created database entries tying it all together</li>
</ul>

<hr />

<h3 id="browse-and-search">Browse and Search</h3>

<p>The last step was to build a nice browse and search interface:</p>

<p><img src="/images/2026-09-27-r3el-catalogue.png" alt="R3el Catalog" /></p>

<p>This part <strong>was</strong> easy, there are so many examples of user interfaces that provide this sort of functionality. Building a simple one was straightforward.</p>

<p><img src="/images/2026-09-27-r3el-tv-show.png" alt="R3el TV Show" /></p>

<hr />

<h3 id="done">Done</h3>

<p>The project is done. R3el is still scanning, but I think it’ll finish by end of day. Early on I wondered if building this was overkill, but realistically, this is the only way I can still think of to tackle a massive data cleanup exercise like this.</p>

<p>And it’s pretty cool, right! :)</p>

<p><img src="/images/2026-09-27-r3el.png" alt="R3el Logo" /></p>

<hr />

<h3 id="links">Links</h3>

<ul>
  <li><a href="https://r3el.osoyalce.com">R3el Project Homepage</a></li>
  <li><a href="https://github.com/NadimGhaznavi/r3el">R3el on GitHub</a></li>
  <li><a href="https://github.com/NadimGhaznavi/r3el/blob/main/CHANGELOG.md">R3el CHANGELOG</a></li>
</ul>]]></content><author><name>Nadim-Daniel Ghaznavi</name><email>nghaznavi@gmail.com</email></author><category term="journal" /><category term="R3el" /><category term="LLM" /><category term="Local AI" /><category term="Media Management" /><category term="Automation" /><category term="Python" /><category term="TMDB" /><category term="Self-Hosted" /><summary type="html"><![CDATA[Our first TV was a small black-and-white thing where, on rare occasions, we’d watch Animal Kingdom. Then Beta and VHS came along, and you could watch them at home! Then came DVDs.]]></summary></entry><entry><title type="html">From Prototype to Product (4/4)</title><link href="https://blog.osoyalce.com/journal/From-prototype-to-product/" rel="alternate" type="text/html" title="From Prototype to Product (4/4)" /><published>2026-09-19T00:00:00+00:00</published><updated>2026-09-19T00:00:00+00:00</updated><id>https://blog.osoyalce.com/journal/From-prototype-to-product</id><content type="html" xml:base="https://blog.osoyalce.com/journal/From-prototype-to-product/"><![CDATA[<p>Fr3d worked: It could talk to the Snake Lab, propose changes to the RNN configuration, run experiments, look at the results, and try again. It proved the idea: a local LLM could act as an autonomous experimenter, tuning a reinforcement-learning system over hundreds of simulations.</p>

<p><img src="/images/2026-09-17-ax3l.png" alt="Ax3l" /></p>

<p>But by the time I was finished with Fr3d, I had also learned exactly how I <em>didn’t</em> want to build the next version.</p>

<p>The code had grown organically around the experiment. Prompts were embedded in the application. Responsibilities overlapped. Logging told me what the program was doing, but not necessarily <em>why</em>. Making changes meant tracing through code that had accumulated along with the ideas.</p>

<p>Fr3d was a successful prototype.</p>

<p>Ax3l was the rewrite.</p>

<hr />

<h2 id="building-the-instrument-panel-first">Building the Instrument Panel First</h2>

<p>With Ax3l, I started from the architectural lessons Fr3d had taught me.</p>

<p>The prompting system became modular and data-driven. Different kinds of interactions with the LLM became distinct prompt classes. Experiment data, prompt construction, simulation control, and reporting became separate concerns.</p>

<p>More importantly, I treated observability as part of the application rather than something I could bolt on later.</p>

<p>Ax3l got its own reporting server and a new event logging system backed by the database. Events were organized into categories and subcategories, and the logs themselves became interactive.</p>

<p>Instead of seeing something like:</p>

<blockquote>
  <p>Simulation completed.</p>
</blockquote>

<p>I could click into that event and inspect the simulation, its configuration, the prompt that produced it, the model’s reasoning, and the resulting data.</p>

<p>The logs stopped being a stream of text.</p>

<p>They became a way of navigating the experiment.</p>

<p>That turned out to matter enormously. Ax3l wasn’t going to run one experiment while I watched. It was going to run hundreds of them, unattended, over days and eventually weeks. If I wanted to understand what the AI was doing, I needed to be able to reconstruct its decisions afterwards.</p>

<hr />

<h2 id="then-i-wanted-to-see-the-snake">Then I Wanted to See the Snake</h2>

<p>At some point, rows of numbers stopped being enough.</p>

<p>The Snake Lab already knew when a game had achieved a new high score, so I modified it to capture the board state at that moment and store it along with the simulation results.</p>

<p>I then exposed that snapshot through the Snake Lab’s ZeroMQ API.</p>

<p>Now Ax3l could retrieve the actual board corresponding to the best game from a simulation and display it in the report.</p>

<p>Suddenly, a score of 47 wasn’t just <code class="language-plaintext highlighter-rouge">47</code>.</p>

<p>There was the snake.</p>

<p>You could see the path it had taken, how long it had become, where the food was, and the state of the board when it achieved that score.</p>

<p>It was a small addition technically, but it changed the personality of the project. The experiment wasn’t just producing database records anymore. There was a little AI snake in there, getting better at its job.</p>

<p>No snakes were harmed in this experiment, although many virtual snakes made questionable life choices!</p>

<hr />

<h2 id="is-it-actually-getting-better-plots">Is It Actually Getting Better? Plots!!!</h2>

<p>A high score alone is a terrible way to answer that question.</p>

<p>Reinforcement learning is noisy. A configuration can get lucky. One spectacular run doesn’t necessarily mean the underlying system has improved.</p>

<p>So I added a histogram showing the distribution of scores across simulations.</p>

<p>Then I added an overlay.</p>

<p>The histogram for the oldest half of the experiment is drawn over the histogram for all of the data. Because the older dataset is a subset of the complete one, its bars can only be equal to or shorter than the corresponding bars for the full experiment.</p>

<p>If Ax3l isn’t accomplishing anything, the additional results should simply pile onto roughly the same distribution.</p>

<p>But if the experiment is working, something much more interesting happens.</p>

<p>The distribution starts creeping to the right.</p>

<p>You can <em>see</em> the experiment learning its way into better regions of the configuration space.</p>

<p>That visualization became much more useful to me than watching a single high-score number tick upwards.</p>

<hr />

<h2 id="from-a-38-to-53-high-score">From a 38 to 53 High Score</h2>

<p>The original working Snake configuration could achieve a high score of about 38.</p>

<p>I knew from my previous experiments that the RNN could do better than that. The question was whether an LLM, given the right tools, history, and experimental framework, could systematically find those better configurations.</p>

<p>At the time of writing, Ax3l has passed 500 simulations.</p>

<p>The current high score is <strong>53</strong>.</p>

<p>More importantly, the score distribution has moved.</p>

<p>Ax3l isn’t simply throwing darts until one happens to hit 53. Across the experiment, the population of results has shifted towards better-performing configurations.</p>

<p>And the experiment is still running.</p>

<hr />

<h2 id="live-experiment-data-snake-web">Live Experiment Data: Snake Web</h2>

<p>There was one final problem.</p>

<p>Ax3l’s reporting server was excellent when I was sitting on my own network, but I wanted the experiment itself to be visible publicly.</p>

<p>I didn’t want to expose the production database or application server to the Internet just to publish some graphs.</p>

<p>So I built one last small project: <strong>Snake Web</strong>.</p>

<p>Every half hour, Snake Web queries the experiment database and generates a set of CSV files containing the data needed by a static GitHub Pages site. Those files are published, and the public site updates with the latest state of the experiment.</p>

<p>The high scores are there.</p>

<p>The histograms are there.</p>

<p>The individual simulation results are there.</p>

<p>Even the seed rotations are visible. When Ax3l moves to a new random seed and performance suddenly drops, the graph faithfully falls off a cliff and begins climbing again.</p>

<p>It’s not a polished-up retrospective showing only the successful bits.</p>

<p>It’s the experiment, scars and all.</p>

<hr />

<h2 id="watching-the-experiment">Watching the Experiment</h2>

<p>And with that, I think the project is finished.</p>

<p>Not because there aren’t another hundred things I could add. There always are.</p>

<p>It’s finished because the system I originally wanted now exists.</p>

<p>There is a dedicated reinforcement-learning laboratory that runs controlled Snake experiments and records their results.</p>

<p>There is an autonomous LLM agent that examines those results, reasons about them, chooses new configurations, and submits further experiments.</p>

<p>There is an observability layer that lets me inspect what it did and why.</p>

<p>There is a public pipeline that publishes the experiment’s progress without exposing the machinery running underneath it.</p>

<p>And somewhere inside all of that architecture, databases, ZeroMQ messages, prompts, PyTorch tensors, CSV files, and graphs is a snake that used to score 38.</p>

<p>Now it scores 53.</p>

<p>That’s a pretty good place to stop.</p>

<hr />

<h2 id="links">Links</h2>

<ul>
  <li><a href="https://ax3l.osoyalce.com">Ax3l Project</a></li>
  <li><a href="https://snakeweb.osoyalce.com">Ax3l’s Live Experiment Site</a></li>
  <li><a href="https://snakelabserver.osoyalce.com">Snake Lab Server</a></li>
</ul>

<h3 id="blog-series">Blog Series</h3>

<ul>
  <li><a href="/journal/Local-LLM-writes-haikus/">Part 1/4 of this Blog Series - Local LLM writes Haikus</a></li>
  <li><a href="/journal/The-evolution-of-SnakeLab/">Part 2/4 of this Blog Series - The Evolution of Snake Lab</a></li>
  <li><a href="/journal/Prompt-engineering-101">Part 3/4 of this Blog Series - Prompt Engineering 101</a></li>
  <li>Part 4/4 of this Blog Series - This post</li>
</ul>]]></content><author><name>Nadim-Daniel Ghaznavi</name><email>nghaznavi@gmail.com</email></author><category term="journal" /><category term="AI" /><category term="LLM" /><category term="Qwen" /><category term="llama.cpp" /><category term="GGUF" /><summary type="html"><![CDATA[Fr3d worked: It could talk to the Snake Lab, propose changes to the RNN configuration, run experiments, look at the results, and try again. It proved the idea: a local LLM could act as an autonomous experimenter, tuning a reinforcement-learning system over hundreds of simulations.]]></summary></entry><entry><title type="html">Prompt Engineering 101 (3/4)</title><link href="https://blog.osoyalce.com/journal/Prompt-engineering-101/" rel="alternate" type="text/html" title="Prompt Engineering 101 (3/4)" /><published>2026-09-12T00:00:00+00:00</published><updated>2026-09-12T00:00:00+00:00</updated><id>https://blog.osoyalce.com/journal/Prompt-engineering-101</id><content type="html" xml:base="https://blog.osoyalce.com/journal/Prompt-engineering-101/"><![CDATA[<p>The <a href="https://snakelabserver.osoyalce.com">Snake Lab</a> had become a proper server, with a ZeroMQ API, configuration validation, a database, and a way to run experiments independently of the client. The obvious next question was:</p>

<p><img src="/images/2026-09-12-fr3d.png" alt="Fr3d" /></p>

<p>What happens if I let an LLM drive it?</p>

<p>That became <strong>Fr3d</strong>.</p>

<p>Fr3d was my first attempt at building an autonomous agent around Snake Lab. The basic idea was fairly simple. Give the model enough information about the experiment, show it the current configuration and the results of previous runs, then ask it to choose a new configuration that might perform better.</p>

<p>Snake Lab would run the experiment.</p>

<p>Fr3d would look at the result.</p>

<p>Then it would try again.</p>

<p>In other words, I had finally connected the little local LLM experiment from my previous post to something that could actually do work.</p>

<hr />

<h2 id="giving-the-ai-an-experiment">Giving the AI an experiment</h2>

<p>I started building prompts that explained the Snake experiment to the model.</p>

<p>The prompt described the neural network, the available parameters, their valid ranges, the current configuration, and whatever experimental history seemed relevant. Fr3d would then choose a parameter value and submit a new experiment through the Snake Lab API.</p>

<p>The whole thing formed a loop:</p>

<p><strong>prompt → LLM → configuration → Snake Lab → result → prompt</strong></p>

<p>And surprisingly quickly, it worked.</p>

<p>Not beautifully.</p>

<p>But it worked.</p>

<p>I already had a known working Snake configuration that could produce a high score of around 38, and I knew from months of manually experimenting with the RNN that better configurations existed. So Fr3d had a real problem to solve.</p>

<p>This wasn’t just asking an LLM to produce plausible-looking JSON.</p>

<p>It had to make decisions against actual experimental results.</p>

<hr />

<h2 id="a-brief-experiment-with-vision">A brief experiment with vision</h2>

<p>Around this time I took a short detour into <strong>Qwen 2.5 VL</strong>, the vision-language version of the model.</p>

<p>The idea was appealing.</p>

<p>Instead of describing everything numerically, perhaps I could show the model plots or even images from the Snake experiments and let it reason visually about what was happening. It worked technically, but I quickly ran into a more important problem.</p>

<p>The vision model simply wasn’t as good at reasoning as Qwen 3.5. For this particular application, that mattered much more than vision.</p>

<p>Qwen 3.5 also exposed its reasoning process, which turned out to be incredibly useful while developing the prompts. I could see what the model thought I was asking it to do. And sometimes it was very clearly telling me:</p>

<p><em>“This prompt is nonsense.”</em></p>

<p>Not literally, perhaps, but effectively.</p>

<p>I would ask it to choose an unused parameter value and then discover, through its reasoning, that every valid value had already been used. Or I would omit some piece of experimental history that the model obviously needed. Or I would give it information in a structure that made perfect sense to me, but left the model struggling to determine which values were actually comparable.</p>

<p>Watching the reasoning made those failures visible.</p>

<p>That was probably my first real lesson in prompt engineering. A bad answer isn’t necessarily a model problem.Sometimes you’ve simply built a bad question.</p>

<hr />

<h2 id="prompt-engineering-is-software-engineering">Prompt engineering is software engineering</h2>

<p>Before Fr3d, I had mostly thought about prompts as instructions. By the end of the project I was thinking about them much more like interfaces.</p>

<p>The model can only reason over the information you give it. The structure of that information matters. The terminology matters. The constraints matter. The history you include matters.</p>

<p>If you tell the model to select a value from a search space, it needs to know the search space. If it must avoid previously tested values, it needs a reliable record of those values. If results from different seeds aren’t directly comparable, the prompt needs to make that distinction clear. And if the prompt contains contradictory requirements, the LLM is going to have exactly the same problem a programmer would have working from a contradictory specification.</p>

<p>The prompts became part of the system architecture.</p>

<hr />

<h2 id="the-prototype-problem">The prototype problem</h2>

<p>After about a week, Fr3d was working.</p>

<ul>
  <li>It could talk to Snake Lab.</li>
  <li>It could submit experiments.</li>
  <li>It could receive results.</li>
  <li>It could reason about those results and choose another configuration.</li>
  <li>It was an actual end-to-end autonomous experiment loop.</li>
</ul>

<p>And the code was horrible.</p>

<p>Not because any individual part was especially complicated. It had simply grown organically while I was figuring out what the system needed to be and how to work with the low level code.</p>

<ul>
  <li>Prompts were embedded directly in application code.</li>
  <li>Experiment-specific logic was tangled together with generic LLM-server logic.</li>
  <li>There was duplication.</li>
  <li>State was scattered around.</li>
  <li>Tracing what had happened during an experiment meant reading through a collection of log files and executing ad-hoc sQL.</li>
  <li>Changing one part of the process meant carefully following dependencies through a fairly monolithic program.</li>
</ul>

<p>It had reached that familiar prototype stage where the software technically worked, but understanding *why- it worked was becoming increasingly difficult. And that was when I realized something else I had gotten wrong.</p>

<hr />

<h2 id="reporting-should-have-been-a-first-class-feature">Reporting should have been a first-class feature</h2>

<p>I had treated logging and reporting as things I could add around the experiment. That was backwards. For an autonomous experimental system, observability is part of the experiment. I needed to be able to answer basic questions easily:</p>

<ul>
  <li>What did the model see?</li>
  <li>What prompt produced this decision?</li>
  <li>What was its reasoning?</li>
  <li>What configuration did it submit?</li>
  <li>What was the result?</li>
  <li>What changed from the previous experiment?</li>
  <li>Which configuration is currently considered the best?</li>
</ul>

<p>A pile of log files could technically answer those questions. But technically answering a question and making the system understandable are very different things. The next system needed to be designed around reporting from the beginning.</p>

<hr />

<h2 id="finding-the-boundaries">Finding the boundaries</h2>

<p>Fr3d also taught me where the natural module boundaries were.</p>

<p>Some parts of the system were generic:</p>

<ul>
  <li>talking to the local LLM server</li>
  <li>managing conversations</li>
  <li>submitting requests</li>
  <li>storing state</li>
  <li>handling responses</li>
  <li>recording events</li>
</ul>

<p>Other parts were specific to this particular experiment:</p>

<ul>
  <li>Snake Lab configuration parameters</li>
  <li>parameter ranges</li>
  <li>experimental history</li>
  <li>prompts</li>
  <li>comparison logic</li>
  <li>rules for choosing the next value</li>
</ul>

<p>Those two categories had become tangled together in Fr3d. Once I could see that distinction clearly, the architecture of the next version became much more obvious.</p>

<ul>
  <li>The prompts needed to become data-driven.</li>
  <li>Prompt construction needed its own abstraction.</li>
  <li>Reporting needed to be built into the architecture.</li>
  <li>Experiment-specific logic needed to be separated from the generic machinery that talked to the LLM.</li>
</ul>

<hr />

<h2 id="start-over">Start over</h2>

<p>There is always a temptation with a working prototype to keep repairing it. Refactor this piece. Move that function. Extract a class. Clean up a prompt. Remove some duplication.</p>

<p>I decided not to.</p>

<p>Fr3d had already done its job. It had proven that the idea worked. More importantly, it had shown me where the difficult parts actually were. That is enormously valuable information, and it is information that is very difficult to get by designing everything perfectly in advance. So I kept the lessons and threw away the architecture.</p>

<p>My takeaway from Fr3d was not <em>design everything correctly before you begin</em>. Almost the opposite.</p>

<ul>
  <li>Build the rough prototype.</li>
  <li>Get the entire system working end to end.</li>
  <li>Watch where it hurts.</li>
  <li>Find the places where the responsibilities naturally separate.</li>
  <li>Figure out what information you wish you had been recording all along.</li>
  <li>Then start over.</li>
</ul>

<p>Fr3d was messy. But it worked. And because it worked, I finally knew what I needed to build next.</p>

<hr />

<h2 id="links">Links</h2>

<ul>
  <li><a href="/journal/Local-LLM-writes-haikus/">Part 1/4 of this Blog Series - Local LLM writes Haikus</a></li>
  <li><a href="/journal/The-evolution-of-SnakeLab/">Part 2/4 of this Blog Series - The Evolution of Snake Lab</a></li>
  <li>Part 3/4 of this Blog Series: This post</li>
  <li><a href="/journal/From-prototype-to-product/">Part 4/4 of this Blog Series - From Prototype to Product</a></li>
</ul>]]></content><author><name>Nadim-Daniel Ghaznavi</name><email>nghaznavi@gmail.com</email></author><category term="journal" /><category term="AI" /><category term="LLM" /><category term="Qwen" /><category term="llama.cpp" /><category term="GGUF" /><summary type="html"><![CDATA[The Snake Lab had become a proper server, with a ZeroMQ API, configuration validation, a database, and a way to run experiments independently of the client. The obvious next question was:]]></summary></entry><entry><title type="html">The Evolution of Snake Lab (2/4)</title><link href="https://blog.osoyalce.com/journal/The-evolution-of-SnakeLab/" rel="alternate" type="text/html" title="The Evolution of Snake Lab (2/4)" /><published>2026-09-05T00:00:00+00:00</published><updated>2026-09-05T00:00:00+00:00</updated><id>https://blog.osoyalce.com/journal/The-evolution-of%20SnakeLab</id><content type="html" xml:base="https://blog.osoyalce.com/journal/The-evolution-of-SnakeLab/"><![CDATA[<p>In my last post, I setup a locally hosted LLM running on an old HP Z440 and wrote a crude little conversation loop. I generated random numbers, fed them to the LLM, and asked it to write haikus about them.</p>

<p><img src="/images/2026-09-05-snake-lab.png" alt="Snake Lab Logo" /></p>

<p>It worked.</p>

<p>I had a machine in my basement writing poetry about random numbers.</p>

<p>Naturally, the next question was: <strong>what else can I get this thing to do?</strong></p>

<p>While I was experimenting with the model, I discovered that it knew about Patrick Loeber’s AI Snake Game experiment. That caught my attention because I knew that experiment very well. I had been playing with and modifying that code for quite some time.</p>

<p>And that gave me an idea.</p>

<p>What if, instead of asking the LLM to write haikus, I gave it the results of a Snake simulation?</p>

<p>I could give it the configuration that produced those results, ask it to reason about them, and then ask it to produce a new configuration. I could run that configuration through the simulator, collect the results, feed them back to the LLM, and repeat.</p>

<p>In other words:</p>

<p><strong>Could I get an LLM to optimize another AI?</strong></p>

<p>That idea would eventually become <strong>Fr3d</strong>.</p>

<p>But there was a problem.</p>

<hr />

<h2 id="the-old-ai-snake-lab-project">The old AI Snake Lab Project</h2>

<p>I had followed Patrick Leober’s tutorial, and then I continued. I ported it into <strong>Textual</strong> and integrated the old MatPlotLib plots with integrated <strong>Textual Plot</strong> plots. The application became separate client and server processes. ZeroMQ connected them. Simulation results were stored in a database.</p>

<p>The Textual client could configure a simulation, send that configuration to the server, start a run, and monitor it while it was running.</p>

<p>It was already much more than a Snake game.</p>

<p>But it was still an <em>application</em>. Its architecture had grown around the assumption that a human being was sitting at the client and conducting experiments.</p>

<p>Fr3d changed that assumption.</p>

<hr />

<h2 id="building-the-lab">Building the Lab</h2>

<p>I took the existing Snake code and split off a new, independent project: <strong>Snake Lab Server</strong>.</p>

<p>The basic idea was simple. Snake Lab Server wouldn’t care who was conducting the experiment.</p>

<p>A client would submit a JSON configuration over ZeroMQ. The server would validate it, queue the experiment, run the simulation, collect the metrics, and store the results in MariaDB.</p>

<p>The server itself would run as a systemd service.</p>

<p>That meant an LLM could use it. A Python script could use it. A terminal application could use it. Something I haven’t thought of yet could use it.</p>

<p>The simulator didn’t need to know.</p>

<p>I also ported my Textual client to the new system. I still wanted to be able to submit experiments myself and, more importantly, watch them run.</p>

<p>There is something considerably more satisfying about actually seeing the snake moving around the board than watching numbers scroll past in a log file.</p>

<p>But now the Textual interface was just <strong>one client</strong>.</p>

<p>That was an important distinction.</p>

<hr />

<h2 id="an-experiment-is-more-than-a-high-score">An Experiment Is More Than a High Score</h2>

<p>I also wanted Snake Lab Server to be fairly strict about what constituted an experiment.</p>

<p>The configuration is validated before a simulation is accepted. The configuration used for the run is recorded. The results are recorded. Metrics are collected throughout the simulation, including per-game and per-episode data.</p>

<p>That matters because I wasn’t interested in simply asking:</p>

<blockquote>
  <p>Did this snake get a better score?</p>
</blockquote>

<p>I wanted to be able to ask:</p>

<blockquote>
  <p>What exactly did we run, and what happened?</p>
</blockquote>

<p>If an automated system was eventually going to conduct hundreds of experiments, I needed the answer to that question to survive long after the experiment had finished.</p>

<p>Otherwise I wouldn’t have a laboratory.</p>

<p>I’d have a slot machine.</p>

<hr />

<h2 id="ready-for-the-scientist">Ready for the Scientist</h2>

<p>And that was Snake Lab Server.</p>

<p>It wasn’t Fr3d yet.</p>

<p>There was no autonomous LLM scientist examining results, reasoning about configurations, and deciding what experiment to run next. That part still had to be built.</p>

<p>But now there was somewhere for it to work.</p>

<p>I had started with an AI learning to play Snake. That had evolved into a client/server application for experimenting with it. And now that application had evolved again, into a service designed specifically so that <strong>other software could conduct controlled experiments on the Snake AI</strong>.</p>

<p>The laboratory was ready.</p>

<p>Now I needed to build the scientist.</p>

<hr />

<h2 id="links">Links</h2>

<ul>
  <li>
    <p><a href="https://snakelabserver.osoyalce.com">Snake Lab Server project site</a></p>
  </li>
  <li><a href="/journal/Local-LLM-writes-haikus/">Part 1/4 of this Blog Series</a></li>
  <li>Part 2/4 is this post.</li>
  <li><a href="/journal/Prompt-engineering-101/">Part 3/4 of this Blog Series</a></li>
</ul>]]></content><author><name>Nadim-Daniel Ghaznavi</name><email>nghaznavi@gmail.com</email></author><category term="journal" /><category term="AI" /><category term="LLM" /><category term="Qwen" /><category term="llama.cpp" /><category term="GGUF" /><summary type="html"><![CDATA[In my last post, I setup a locally hosted LLM running on an old HP Z440 and wrote a crude little conversation loop. I generated random numbers, fed them to the LLM, and asked it to write haikus about them.]]></summary></entry><entry><title type="html">Local LLM Writes Haikus (1/4)</title><link href="https://blog.osoyalce.com/journal/Local-LLM-write-haikus/" rel="alternate" type="text/html" title="Local LLM Writes Haikus (1/4)" /><published>2026-08-29T00:00:00+00:00</published><updated>2026-08-29T00:00:00+00:00</updated><id>https://blog.osoyalce.com/journal/Local-LLM-write-haikus</id><content type="html" xml:base="https://blog.osoyalce.com/journal/Local-LLM-write-haikus/"><![CDATA[<p>This is part one of a four part blog series. In this post, I download, install and setup some software to prompt a local LLM to write haikus…</p>

<p>I recently acquired a pair of matching HP Z440 workstations with M4000 GPUs. I did some research and learned that I could run a small local LLM on that hardware. I’ve only ever engaged with LLMs as an end-user, in my browser or in my VS code editor.</p>

<p>So I downloaded the <strong>Qwen3.5 4B</strong> model from Hugging Face and started figuring out how to run it using <code class="language-plaintext highlighter-rouge">llama.cpp</code>.</p>

<hr />

<h2 id="building-the-model">Building the Model</h2>

<p>Getting everything running involved a few pieces.</p>

<p>I downloaded the Qwen model from Hugging Face and the <code class="language-plaintext highlighter-rouge">llama.cpp</code> source code. From there I built the intermediate model and eventually converted it into the <strong>GGUF</strong> format used by <code class="language-plaintext highlighter-rouge">llama.cpp</code>. The process was technical, but straightforward. The longest part wa the 18 Gb download.</p>

<p>Then, I fired up <code class="language-plaintext highlighter-rouge">llama_server</code>, pointed my browser at it and…</p>

<p>There it was.</p>

<p>A web page with a chat box.</p>

<p>I had built myself a chatbot.</p>

<p>That was simultaneously very cool and not particularly interesting.</p>

<p>I already had ChatGPT, which is a <strong>LOT</strong> more knowledgeable and skilled.</p>

<hr />

<h2 id="talking-to-it">Talking to It</h2>

<p>Of course, the first thing I did was start asking it questions.</p>

<p>I asked it some general knowledge questions. It knew historical figures (e.g. Rudolf Steiner), geography, science. Coding…</p>

<p>The answer was: quite a lot.</p>

<p>That was impressive in itself. This wasn’t reaching out to Google or calling some giant AI service on the Internet. The model and everything it knew were sitting right there on my own machine.</p>

<p>Then I asked it about <strong>Patrick Loeber’s AI Snake tutorial</strong>.</p>

<p>It knew what I was talking about.</p>

<p>That got my attention.</p>

<p>Patrick Loeber’s project was familiar territory for me. My own <strong>Snake Lab</strong> project grew out of experimenting with an AI learning to play Snake, so suddenly I wasn’t asking the model trivia questions anymore. I could talk to it about something I actually knew.</p>

<p>And it could talk back intelligently.</p>

<p>That was cool.</p>

<hr />

<h2 id="now-what">Now What?</h2>

<p>The problem was that I still had a chatbot.</p>

<p>I could ask it questions. It could answer them. That was impressive, but it wasn’t really what interested me.</p>

<p>I wanted to use the LLM <strong>from code</strong>.</p>

<p>So I wrote a crude conversation loop.</p>

<p>There wasn’t any grand plan at this point. I just wanted a program that could send a prompt to the model, get a response back, and do something with it.</p>

<p>Which raised an important engineering question:</p>

<p><strong>What should I ask it to do?</strong></p>

<p>Apparently my answer was:</p>

<p>Write haikus about numbers.</p>

<hr />

<h2 id="random-haikus">Random Haikus</h2>

<p>The program generated a random number and sent it to the LLM with a prompt asking it to write a haiku about that number.</p>

<p>The LLM wrote the haiku.</p>

<p>The program wrote the result to a log file.</p>

<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>A new age starts today
Apollo touches the moon's face
New world begins to shine

1970
</code></pre></div></div>

<p>Then it generated another number.</p>

<p>And another.</p>

<p>And another.</p>

<p>Soon I had a locally hosted Large Language Model sitting on my computer, contemplating random integers and writing poetry about them.</p>

<p>It wasn’t exactly useful.</p>

<p>But it worked.</p>

<p>The important part wasn’t the haiku. The important part was the loop:</p>

<ol>
  <li>My program generated some input.</li>
  <li>It constructed a prompt.</li>
  <li>It sent the prompt to the LLM.</li>
  <li>The LLM generated a response.</li>
  <li>My program captured the response.</li>
  <li>The process repeated.</li>
</ol>

<p>For the first time, I wasn’t really <strong>talking to an AI</strong>.</p>

<p>I had put an AI <strong>inside a program</strong>.</p>

<hr />

<h2 id="conclusion">Conclusion</h2>

<p>My first experiment with a locally hosted LLM produced absolutely nothing of practical value.</p>

<p>It wrote haikus about random numbers.</p>

<p>But that little experiment answered the question I actually cared about.</p>

<p>I could run an LLM locally. I could communicate with it from my own software. I could give it information, get a response, record that response and repeat the process without sitting in front of a chat window.</p>

<p>At that point I didn’t have <strong>Ax3l</strong>.</p>

<p>I didn’t even have the idea fully formed.</p>

<p>I had a loop.</p>

<p>And a log file full of haikus.</p>

<hr />

<h2 id="links">Links</h2>

<ul>
  <li><a href="/journal/The-evolution-of-SnakeLab/">Part 2/4 of this Blog Series</a></li>
</ul>]]></content><author><name>Nadim-Daniel Ghaznavi</name><email>nghaznavi@gmail.com</email></author><category term="journal" /><category term="AI" /><category term="LLM" /><category term="Qwen" /><category term="llama.cpp" /><category term="GGUF" /><summary type="html"><![CDATA[This is part one of a four part blog series. In this post, I download, install and setup some software to prompt a local LLM to write haikus…]]></summary></entry><entry><title type="html">From Seed to Signal: AI FX</title><link href="https://blog.osoyalce.com/journal/AIFX-signal/" rel="alternate" type="text/html" title="From Seed to Signal: AI FX" /><published>2026-05-18T00:00:00+00:00</published><updated>2026-05-18T00:00:00+00:00</updated><id>https://blog.osoyalce.com/journal/AIFX-signal</id><content type="html" xml:base="https://blog.osoyalce.com/journal/AIFX-signal/"><![CDATA[<p>In the first post, AI FX was still mostly a seed: a repo, a PyPI package, and an idea about using automated FOREX tooling as a learning lab. Since then, the seed has grown roots. The project now has a working broker/client architecture, live OANDA data flowing into an in-memory database layer, published over ZeroMQ, and consumed by a Qt GUI that renders live candlestick data. It is still early, but it has crossed an important line: AI FX is no longer just scaffolding. It is a running system.</p>

<p><img src="/images/2026-05-18-aifx-live-eurcad.png" alt="AI FX live EUR/CAD view" /></p>

<hr />

<h1 id="the-shape-of-the-system">The Shape of the System</h1>

<p>The current architecture has three main pieces:</p>

<ul>
  <li>A broker process that owns the core data flow</li>
  <li>An OANDA manager that talks to the external Oanda API</li>
  <li>A Qt client that visualizes the feed and shows link latency information</li>
</ul>

<p>The broker is the center of gravity. It fetches market data through OANDA, stores candle data in an in-memory SQLite database, and publishes updates to clients through ZeroMQ.</p>

<p><img src="/images/2026-05-18-aifx-architecture.png" alt="AI FX broker/client architecture" /></p>

<p>The Qt client connects to the broker, subscribes to a candlestick topic, and updates the interface as new data arrives. The result is a small live market console: recent candles in a table, a candlestick chart underneath, instrument selection, broker and OANDA latency.</p>

<p>That last part matters. This project is not just about making a chart move. It is about building a system that can observe itself.</p>

<hr />

<h1 id="why-the-broker-exists">Why the Broker Exists</h1>

<p>One thing I wanted to avoid was turning the GUI into the trading system.</p>

<p>The GUI is useful for making selections, for rendering, resizing, tables, charts, buttons and more, but that is not where the data feed should be managed.</p>

<p>The broker gives the project a clean separation:</p>

<ul>
  <li>OANDA access stays behind a manager</li>
  <li>Market data is normalized before clients see it</li>
  <li>The in-memory database acts as a local working set</li>
  <li>ZeroMQ becomes the transport layer between the broker and client processes</li>
  <li>Clients can subscribe without owning the data pipeline</li>
</ul>

<p>That separation is already paying off. The Qt client can disconnect and reconnect without being the source of truth. The broker can keep doing broker things. The GUI can keep doing GUI things.</p>

<hr />

<h1 id="the-gui-milestone">The GUI Milestone</h1>

<p>The latest milestone is the Qt interface. I was initially going to use Textual for the client, but when I saw Plotly’s candlestick plots I decided I needed that.
The AIFX client uses <a href="https://doc.qt.io/qtforpython-6/">Qt for Python</a>, renders <a href="https://plotly.com/python/basic-charts/">Plotly charts</a> in a <a href="https://doc.qt.io/qtforpython-6/PySide6/QtWebEngineCore/index.html">QtWebEngineCore</a>, and</p>

<p>The client now opens with historical candles already loaded, so the screen does not start empty while waiting for fresh data. Instrument switching works. The recent candle table updates. The chart updates. Status labels show broker and OANDA call latency.</p>

<p>That may sound cosmetic, but it changes the feel of the project.</p>

<p>A console log tells you the machinery is alive. A live chart makes it obvious.</p>

<p>It also surfaces problems faster. Missing candles, stale subscriptions, latency spikes, formatting issues, and feed timing quirks all become visible. The GUI is not just eye candy. It is a diagnostic cockpit.</p>

<hr />

<h1 id="small-candles-fast-feedback">Small Candles, Fast Feedback</h1>

<p>AI FX is currently focused on five-second candles.</p>

<p>That is a small enough interval to make the system feel alive, but still structured enough to work with as discrete data. Each candle becomes a compact summary of a tiny slice of market activity: open, high, low, close, volume, and timestamp.</p>

<p>For now, the project is still in the infrastructure phase. The work is not about pretending the system is ready to trade. It is about building the rails:</p>

<ul>
  <li>Fetch data reliably</li>
  <li>Store it cleanly</li>
  <li>Publish it predictably</li>
  <li>Display it clearly</li>
  <li>Test it without guessing</li>
</ul>

<p>The trading logic can come later. The wiring and plumbing come first.</p>

<hr />

<h1 id="tests-releases-and-the-unromantic-bits">Tests, Releases, and the Unromantic Bits</h1>

<p>A surprising amount of progress has come from the unglamorous parts.</p>

<p>There are unit tests now for core data classes like instruments and candles. There is a release process. There are linting and typing checks. There is a fresh-install smoke test path to make sure the package works outside the development tree.</p>

<p>That matters because this kind of project can become a spaghetti aquarium very quickly.</p>

<p>Market data, networking, async loops, GUI events, serialization, plotting, packaging, and API calls all want to tangle themselves together. Tests and release discipline are the small fences that keep the goats out of the server room.</p>

<hr />

<h1 id="still-a-small-machine">Still a Small Machine</h1>

<p>AI FX is still a small machine.</p>

<p>It is not an institutional trading platform. It is not a magic money machine. It is not trying to outrun colocated infrastructure or pretend retail latency does not exist.</p>

<p>But it is becoming a useful lab.</p>

<p>The original idea was to build something small enough to understand and flexible enough to experiment with. That is still the goal.</p>

<p>The seed is no longer just in the ground.</p>

<p>There is a signal now.</p>]]></content><author><name>Nadim-Daniel Ghaznavi</name><email>nghaznavi@gmail.com</email></author><category term="journal" /><category term="AI" /><category term="FOREX" /><category term="Qt" /><category term="OANDA" /><category term="ZeroMQ" /><summary type="html"><![CDATA[In the first post, AI FX was still mostly a seed: a repo, a PyPI package, and an idea about using automated FOREX tooling as a learning lab. Since then, the seed has grown roots. The project now has a working broker/client architecture, live OANDA data flowing into an in-memory database layer, published over ZeroMQ, and consumed by a Qt GUI that renders live candlestick data. It is still early, but it has crossed an important line: AI FX is no longer just scaffolding. It is a running system.]]></summary></entry><entry><title type="html">Simulating the Power Grid</title><link href="https://blog.osoyalce.com/journal/The-Grid/" rel="alternate" type="text/html" title="Simulating the Power Grid" /><published>2026-04-26T00:00:00+00:00</published><updated>2026-04-26T00:00:00+00:00</updated><id>https://blog.osoyalce.com/journal/The-Grid</id><content type="html" xml:base="https://blog.osoyalce.com/journal/The-Grid/"><![CDATA[<p>A potential client pointed me to <a href="https://lfenergy.org/empowering-the-energy-transition-through-fast-and-flexible-network-simulation/">Empowering the Energy Transition Through Fast and Flexible Network Simulation</a>. The article describes an Open Source project, <a href="https://power-grid-model-ds.readthedocs.io/en/stable/">Power Grid Model DS</a> (<code class="language-plaintext highlighter-rouge">pgm-ds</code>), where the <em>DS</em> is <em>Data Science</em>.</p>

<p><img src="/images/2026-04-26-pgm-ds-logo.png" alt="PGM DS Logo" /></p>

<p>The <code class="language-plaintext highlighter-rouge">pgm-ds</code> project extends the <a href="https://power-grid-model.readthedocs.io/en/stable/">Power Grid Model</a>. From their docs, <em>“power-grid-model is a library for steady-state distribution power system analysis.”</em> Its model includes components for:</p>

<ul>
  <li>Node</li>
  <li>Branch</li>
  <li>Appliance</li>
  <li>Sensor</li>
  <li>Fault</li>
  <li>Regulator</li>
</ul>

<p>-and more. It’s used to model power flow, state estimation, and short circuits.</p>

<p>The <em>Power Grid Model DS</em> builds on top of this to provide a clean and easy-to-use API to model and simulate the behaviour of the grid. <em>“This is aimed at building data science software applications related to or using the power-grid-model project, such as network analyses and simulations. It defines a Grid dataclass which manages the consistency of the complete network and allows for extensions of the Power Grid Model datastructure.”</em></p>

<hr />

<h1 id="running-the-demo">Running the Demo</h1>

<p>The mechanics were straightforward; the meaning took effort.</p>

<p>The <code class="language-plaintext highlighter-rouge">pgm-ds</code> <strong>is</strong> easy to use. It was relatively trivial to take their Jupyter notebook-style docs and create a script to run the demo. Understanding what I was looking at took more work.</p>

<p>The package includes a visualization of the grid using a <a href="https://flask.palletsprojects.com/en/stable/">Flask</a> WSGI frontend that you can easily view with your web browser. The graphics use <a href="https://dash.plotly.com/cytoscape">Dash Cytoscape</a>, a slick and powerful graph visualization engine that’s perfect for this sort of visualization.</p>

<p>While the picture was pretty, the informative data was presented on the console in the form of tables. Here are some examples:</p>

<h2 id="nodes">Nodes</h2>

<table>
  <thead>
    <tr>
      <th>id</th>
      <th>u_rated</th>
      <th>node_type</th>
      <th>feeder_branch_id</th>
      <th>feeder_node_id</th>
      <th>u</th>
      <th>x_coor</th>
      <th>y_coor</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <td>1</td>
      <td>10500.0</td>
      <td>0</td>
      <td>64</td>
      <td>61</td>
      <td>0.0</td>
      <td>125.848..</td>
      <td>245.522..</td>
    </tr>
    <tr>
      <td>2</td>
      <td>10500.0</td>
      <td>0</td>
      <td>65</td>
      <td>61</td>
      <td>0.0</td>
      <td>385.322..</td>
      <td>175.70..</td>
    </tr>
    <tr>
      <td>61</td>
      <td>10500.0</td>
      <td>1</td>
      <td>-2147483648</td>
      <td>-2147483648</td>
      <td>0.0</td>
      <td>134.732..</td>
      <td>145.515..</td>
    </tr>
  </tbody>
</table>

<h2 id="lines">Lines</h2>

<table>
  <thead>
    <tr>
      <th>id</th>
      <th>from_node</th>
      <th>to_node</th>
      <th>from_status</th>
      <th>to_status</th>
      <th>feeder_branch_id</th>
      <th>feeder_node_id</th>
      <th>is_feeder</th>
      <th>r1</th>
      <th>x1</th>
      <th>c1</th>
      <th>tan1</th>
      <th>i_n</th>
      <th>i_from</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <td>63</td>
      <td>61</td>
      <td>12</td>
      <td>1</td>
      <td>1</td>
      <td>63</td>
      <td>61</td>
      <td>True</td>
      <td>0.411..</td>
      <td>0.023..</td>
      <td>0.0</td>
      <td>0.0</td>
      <td>110.013..</td>
      <td>0.0</td>
    </tr>
    <tr>
      <td>74</td>
      <td>20</td>
      <td>8</td>
      <td>1</td>
      <td>1</td>
      <td>63</td>
      <td>61</td>
      <td>False</td>
      <td>0.585..</td>
      <td>0.000..</td>
      <td>0.0</td>
      <td>0.0</td>
      <td>227.694..</td>
      <td>0.0</td>
    </tr>
    <tr>
      <td>125</td>
      <td>96</td>
      <td>94</td>
      <td>1</td>
      <td>1</td>
      <td>63</td>
      <td>61</td>
      <td>False</td>
      <td>0.036..</td>
      <td>0.04..</td>
      <td>0.0</td>
      <td>0.0</td>
      <td>362.904..</td>
      <td>0.0</td>
    </tr>
  </tbody>
</table>

<p>## Transformers</p>

<table>
  <thead>
    <tr>
      <th>id</th>
      <th>from_node</th>
      <th>to_node</th>
      <th>from_status</th>
      <th>to_status</th>
      <th>feeder_branch_id</th>
      <th>feeder_node_id</th>
      <th>is_feeder</th>
      <th>u1</th>
      <th>u2</th>
      <th>sn</th>
      <th>tap_size</th>
      <th>uk</th>
      <th>pk</th>
      <th>i0</th>
      <th>p0</th>
      <th>winding_from</th>
      <th>winding_to</th>
      <th>clock</th>
      <th>tap_side</th>
      <th>tap_pos</th>
      <th>tap_min</th>
      <th>tap_max</th>
      <th>tap_nom</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <td>123</td>
      <td>5</td>
      <td>95</td>
      <td>1</td>
      <td>1</td>
      <td>64</td>
      <td>61</td>
      <td>False</td>
      <td>10500.0</td>
      <td>3000.0</td>
      <td>30000000.0</td>
      <td>0.0</td>
      <td>0.203</td>
      <td>100000.0</td>
      <td>0.0</td>
      <td>0.0</td>
      <td>0</td>
      <td>0</td>
      <td>12</td>
      <td>0</td>
      <td>0</td>
      <td>0</td>
      <td>0</td>
      <td>0</td>
    </tr>
    <tr>
      <td>124</td>
      <td>12</td>
      <td>96</td>
      <td>1</td>
      <td>1</td>
      <td>63</td>
      <td>61</td>
      <td>False</td>
      <td>10500.0</td>
      <td>3000.0</td>
      <td>30000000.0</td>
      <td>0.0</td>
      <td>0.203</td>
      <td>100000.0</td>
      <td>0.0</td>
      <td>0.0</td>
      <td>0</td>
      <td>0</td>
      <td>12</td>
      <td>0</td>
      <td>0</td>
      <td>0</td>
      <td>0</td>
      <td>0</td>
    </tr>
  </tbody>
</table>

<hr />

<h1 id="understanding-the-data">Understanding the Data</h1>

<p>Setting up and running the code <strong>was</strong> trivial. Understanding the output and the implications of what the software does was not. My background is as a Linux Systems Admin and Python Programmer; I don’t have a background in electrical engineering.</p>

<p>I started by drilling into the data structures, functions, looking at the clear and concise <a href="https://power-grid-model-ds.readthedocs.io/en/stable/api_reference/power_grid_model_ds.html">API</a>, and doing research.</p>

<p>I had always pictured the grid as a complicated but largely static circuit. What became clear is that it’s better understood as a dynamic physical system, where electrical properties like reactance and reactive power materially influence behavior in ways that aren’t immediately intuitive. The grid is similar to a living, breathing entity.</p>

<p>Right away, there was a value in the demo, <strong>reactance per kilometer</strong>, which also shows up as an attribute of the lines, <code class="language-plaintext highlighter-rouge">x1</code>. Reactance was probably the hardest part to wrap my mind around. I did end up watching some YouTube videos and talking it through with ChatGPT. <strong>Reactive power</strong> is central to understanding why the grid is so difficult to manage. A full explanation is beyond the scope of this post, but it’s an essential concept for appreciating the system’s behavior.</p>

<p>I didn’t drill into every attribute of the system; I was just trying to get a sense of what this package did and achieve an overview of how it mapped into the real world.</p>

<hr />

<h1 id="modeling-the-grid">Modeling the Grid</h1>

<p>When I realized how complicated and intricate the grid actually was, the scale of it jumped a couple of orders of magnitude in my mind. As I mentioned earlier, my background includes Linux, and I’ve worked in Ops teams at places like CIBC, Merrill Lynch, Moneris Solutions, and SiGe Semiconductor. I’ve seen those big dashboards full of technical detail that provide the Ops Teams with a dashboard. Then I googled for a Hydro One (Ontario’s power producer) ops dashboard.</p>

<p><img src="/images/2026-04-26-hydro-one-ops.png" alt="Hydro One Ops" /></p>

<p>While this is only a slice of the system, it illustrates the density and complexity operators deal with in practice.</p>

<p>The <code class="language-plaintext highlighter-rouge">pgm-ds</code> software allows you to dynamically modify the grid, add and remove components and then it reports on faults and failures. It allows you to re-route power through redundant links and more. You can model changes and then try to forecast future demand and the effects of change.</p>

<p>With a system this complex, simulation becomes a natural foundation for machine learning. Tools like <code class="language-plaintext highlighter-rouge">pgm-ds</code> make it possible to generate structured scenarios for forecasting, anomaly detection, and decision support.</p>

<p>Extending and managing the grid is not only non-trivial, it’s also a slow moving project. Building a power plant, a dam, a data center, a factory, or a neighborhood all take time.</p>

<hr />

<h1 id="conclusion">Conclusion</h1>

<p>This field is both technically fascinating and materially important. Tools like this don’t just model infrastructure—they help shape how it evolves. That makes the work both intellectually engaging and meaningful.</p>

<hr />

<h1 id="links">Links</h1>

<h2 id="companies-and-organizations">Companies and Organizations</h2>

<ul>
  <li><a href="https://lfenergy.org/">LF Energy Website</a></li>
  <li><a href="https://www.alliander.com/en/">Alliander</a></li>
</ul>

<h2 id="power-grid-model-ds">Power Grid Model DS</h2>

<ul>
  <li><a href="https://lfenergy.org/empowering-the-energy-transition-through-fast-and-flexible-network-simulation/">Power Grid Model Data Science Toolkit Overview</a></li>
  <li><a href="https://archive.fosdem.org/2025/events/attachments/fosdem-2025-4662-empowering-the-energy-transition-through-fast-and-flexible-network-simulation/slides/238824/FOSDEM_Pr_gNLHXgk.pdf">Slides from a dgm-ds Presentation</a></li>
  <li><a href="https://power-grid-model-ds.readthedocs.io/en/stable/quick_start.html#modifying-the-grid">Quick Start — power-grid-model-ds documentation</a></li>
  <li><a href="https://power-grid-model-ds.readthedocs.io/en/stable/demos/connecting_new_consumer.html">Full example: Connecting a new consumer — power-grid-model-ds documentation</a></li>
</ul>

<h2 id="api-docs">API Docs</h2>

<ul>
  <li><a href="https://power-grid-model-ds.readthedocs.io/en/stable/">Power Grid Model DS</a></li>
  <li><a href="https://power-grid-model.readthedocs.io/en/stable/">Power Grid Model</a></li>
</ul>

<h2 id="github-repos">GitHub Repos</h2>
<ul>
  <li><a href="https://github.com/PowerGridModel/">Power Grid Model umbrella</a>
    <ul>
      <li><a href="https://github.com/PowerGridModel/power-grid-model">power-grid-model</a></li>
      <li><a href="https://github.com/PowerGridModel/power-grid-model-ds">power-grid-model-ds</a></li>
      <li><a href="https://github.com/PowerGridModel/power-grid-model-io">power-grid-model-io</a></li>
    </ul>
  </li>
  <li>
    <p><a href="https://github.com/plotly/dash-cytoscape">Dash Cytoscape</a></p>
  </li>
  <li><a href="Email address">powergridmodel+subscribe@lists.lfenergy.org:mailto</a> for subscribing to the the Power Grid Model mailing list</li>
</ul>

<h2 id="core-concepts">Core Concepts</h2>

<ul>
  <li><a href="https://www.rustworkx.org/tutorial/introduction.html">RustworkX</a></li>
  <li><a href="https://www.youtube.com/watch?v=2mD3UbSW7ho">Reactive Power Explained</a></li>
</ul>]]></content><author><name>Nadim-Daniel Ghaznavi</name><email>nghaznavi@gmail.com</email></author><category term="journal" /><category term="pgm-ds" /><category term="Power Grid Modeling Data Science" /><category term="Simulation" /><summary type="html"><![CDATA[A potential client pointed me to Empowering the Energy Transition Through Fast and Flexible Network Simulation. The article describes an Open Source project, Power Grid Model DS (pgm-ds), where the DS is Data Science.]]></summary></entry></feed>