IoT Platform Infrastructure on GCP
End-to-end infrastructure and release automation for the iMatrix sensor platform, from Terraform-managed GCP and GKE to tag-gated releases with one-step rollback.
At Sirin Software I was the DevOps engineer for iMatrix, an IoT platform built around sensor gateways that report to the cloud and receive firmware updates over the air. The cloud side had seven services covering device traffic, customer-facing APIs, background processing, and the web UI. For roughly ten months I owned the infrastructure and the release process, which came to about 200 commits across Terraform, Jenkins pipelines, and Kubernetes manifests.
Releases with a way back
I wrote the Jenkins pipelines for every service, split by risk. Development deploys run automatically over SSH. Production builds only run from the tagged release line and abort if the head commit has no git tag, so nothing ships without a version number. Images are pushed to Artifact Registry with both the version tag and latest, rollout is a kubectl set image followed by rollout status, and rollback is a separate job that runs rollout undo. Later additions include email notification on successful prod releases and a system-test stage on dev.
The business effect is that releasing stopped being an event. Any engineer could ship a tagged version, and a bad release cost a rollback job instead of an outage window with a device fleet waiting on the cloud.
Firmware publishing without hand-carried files
Firmware used to reach the OTA service by someone uploading a build. I replaced that with scripts that run in the pipeline: they extract the version from the firmware’s C headers, or from the artifact filename for the other device product lines, map the build to the right product id, and upload it to the OTA API with a beta flag.
Version numbers now come from the source tree instead of someone’s memory, and a firmware build lands where devices can fetch it as part of the same job that compiled it.
The GCP footprint in Terraform
I declared the infrastructure in Terraform: project APIs, service accounts and their IAM bindings, Artifact Registry, Cloud Storage buckets, static internal and external addresses, the network, an Autopilot GKE cluster with private nodes behind Cloud NAT, and the VMs running Redis, InfluxDB, and the ELK stack. Bootstrap scripts before and after apply handle service-account creation, API enablement, and installing and configuring Redis and InfluxDB on the new VMs.
An environment can be rebuilt from the repository instead of from documentation and memory, which is also the disaster-recovery answer for everything except the data itself.
Device-facing services on GKE
I wrote the deployment manifests for all seven services, plus a Helm chart for the API. Device traffic gets UDP load balancers, health checks, an HPA, and a non-preempting priority class, so the telemetry path is the last thing the cluster sacrifices under pressure. Internal load balancers keep service-to-service traffic off the public internet.
For a hardware company this is the part customers feel: telemetry keeps arriving and OTA keeps serving while the application services are being deployed or scaled.
Development on VMs, logs in ELK
Not everything ran on the cluster. The development environment was a set of VM instances that I managed: Docker on the hosts, per-service deploy jobs, and the web UI built and served from the VM itself. Developers got a place to break things without touching production, and it cost a few VMs instead of a second GKE environment.
I also ran the ELK stack and wrote the log ingestion and field extraction myself, including runtime fields for the messages that would not parse into anything useful on their own. Device and service logs ended up in one searchable place, so triage started with a query instead of SSH sessions into several machines.
Side websites
Alongside the platform I administered the company’s smaller selling websites: Apache and Nginx virtual hosts and their SSL certificates. That work is unglamorous, but those sites are where customers first meet the product, and an expired certificate is a cheap way to look unreliable.
Backups and alerting
I wrote the backup and restore scripts and documented both directions. CloudSQL dumps run over TLS with client certificates and cover the three application databases; InfluxDB backups use the portable format over an SSH tunnel with a control socket, since the time-series VM is not reachable any other way. Monitoring is declared in Terraform too: uptime checks and alert policies across the core services, disk-space alerts on the stateful VMs, and an email notification channel.
Restores are a written procedure rather than a hope, and the common failures, a dead service or a disk filling up on the telemetry database, page the team before customers notice.