I Built a Small DevOps and AI Lab at Home

Why did I need one?

About four months ago, I decided to set up a small Linux server at home.

Linux devices were not new to me. At work, we often deploy the applications we build to Linux devices at customer sites. Connecting over SSH, checking services, reading logs, and solving network problems are all part of my job.

I work as a full-stack developer, but my work does not stop at the frontend and backend. I also deal with installing applications and keeping them running.

I also wanted to test small LLMs in my own environment. Instead of connecting to a remote service for every idea, I wanted to see how a model behaved, how many resources it used, and how it could connect to a simple application.

So I set up a small environment where I could work on both my applications and local AI experiments.

I had a small device that could run Linux. It was quiet, used little power, and could stay on all the time. That was enough to get started.

The first setup

The first Linux image I installed did not work as expected. It wanted a monitor and a graphical interface. I wanted to manage the device entirely remotely.

I installed a simpler image, connected the device to my local network, and made the first SSH connection. I have not needed a monitor or keyboard since then.

I updated the system, configured the users and access rules, and started moving some of my smaller applications onto it.

At first, I only wanted to run a few services. Then I started testing small local models and the applications connected to them in the same environment.

Wi-Fi took the most time

Wi-Fi worked while the device was running. After a restart, it sometimes failed to reconnect.

I thought I had fixed it several times. Then I unplugged Ethernet and started the device from a fully powered-off state. Wi-Fi failed again.

Part of the problem was how I was testing it. When Ethernet was connected, a network event also triggered the Wi-Fi connection. I thought my latest configuration change had fixed the problem.

A real cold boot test showed something different.

I added a small watchdog that checks the connection. If Wi-Fi drops, it scans again and tries to reconnect.

For the final test, I booted the device without Ethernet. It connected on its own.

My main mistake was not testing under the right conditions. A configuration looking correct was not enough.

Small problems, real experience

Other unexpected problems came up during the setup.

One day, package installation started failing with certificate and signature errors. The network and package repositories were fine. The system clock was wrong.

On another day, I lost SSH access while changing security settings. The device was next to me, so I could recover it. Making the same mistake on a remote production device would have been much more stressful.

I also ran into the storage limits of a small device. A service with frequent writes turned the SD card into a bottleneck. Model files were not small either. Where I kept each type of data started to matter.

I moved the data to more suitable storage. Then I checked not only that the service started, but also that it still worked after a reboot.

These problems were not completely new to me. The difference was that I had built this system myself, so I could follow each problem through every layer.

How do I run my applications?

The setup depends on the project.

I manage some small services directly with systemd. For other projects, I build Docker images and run them in containers.

Systemd is enough for a simple service. Docker is more convenient when a project has several dependencies or when I want to reproduce the same environment later.

I check the services with health endpoints, inspect their logs, and test whether the applications return after the device restarts.

Self-hosting showed me again that the work does not end when the code is deployed. The application must stay up for days and recover when something fails.

Testing small local models

Local LLM experiments became an important part of this setup.

The device does not have a powerful GPU. I do not try to run large models on it. I use smaller, quantized models instead. The goal is not to get the fastest answer. I want to see what is possible with limited CPU and memory.

I start a model and connect it to my applications through a local API. I watch its prompt behavior, response time, and memory use. Sometimes a model is not good enough for the task. Sometimes it works well for a small job.

Keeping the data on the device is also useful. I can make small RAG tests with personal files or try simple agent flows without sending the data elsewhere.

This device does not replace cloud services. That is not my goal. It is a small AI lab where I can test an idea and understand the infrastructure behind the model.

Safe external access

I did not want to expose services on my home network directly to the internet.

Instead of opening ports on the router, I used Cloudflare Tunnel. The connection starts from the Linux device and goes out to Cloudflare. I also keep management pages behind access control.

Nginx handles the traffic inside the network and sends each request to the right application. This gave me a real environment for working with domains, TLS, reverse proxies, and access control.

It is easy to get lost in the details. A single / in an Nginx setting can make a running service return 404 from the outside. Sometimes the bug I spend hours looking for really is one character.

How did I use Codex?

I did not use Codex only to generate commands.

When something failed, we reviewed the logs, listed possible causes, prepared small tests, and continued based on the results.

Codex helped me interpret Linux logs, investigate network problems, prepare Docker and service configurations, organize local model tests, and document the changes we made.

The first guess was not always correct.

We tried several theories while investigating the Wi-Fi issue. Some looked reasonable but failed in the real test. What I observed on the device, combined with what Codex found in the logs, helped us set up the right test.

Codex was most useful during investigation and testing. Whether I was testing a model or fixing the infrastructure, we could continue without losing the earlier steps.

I still make the final checks. After a change, I read the logs, test the service, and restart the device when needed. I do not accept “it should work” as a result.

Codex does not manage the system for me. It speeds up the investigation and points out things I may have missed.

What did this project teach me?

I was already familiar with deploying applications to Linux devices through my professional work.

Because this system is at home, I can change and rebuild it freely. I have worked directly with everything from networking and storage to containers and service management.

Over the last four months, I have had plenty of hands-on practice with Linux, Docker, systemd, networking, secure tunnels, local inference, and application operations.

It also reminded me that deployment is not the last step.

The network can drop. Storage can slow down. A container can stop. A model can fit in memory but still be too slow to use. A service can start but fail to receive requests. Sometimes everything really does work as expected.

The small Linux device in the corner of my desk is no longer just a server. It is a DevOps and AI lab where I run my applications, test small models, and rebuild what I break.

For me, breaking the system and getting it running again has been the fun part.