AcademicResearch · containers & edge computing
Container virtualisation frameworks in Edge-to-Cloud and serverless environments
Bachelor’s thesis (9.5/10). Extended the DAGonStar workflow engine with Docker, Kubernetes (k3s/k0s), Nomad and Apptainer executors and a cross-runtime data-staging layer, then measured performance, overhead and energy on a real Edge–Fog–Cloud testbed.
- Role
- Author — design, implementation, experimentation and evaluation
- Context
- Universidad Carlos III de Madrid
- Period
- 2025 – 2026
01Overview
Edge-to-Cloud infrastructures mix ARM devices, desktop machines and managed cloud services, with no shared filesystem between them. My thesis extended DAGonStar — a DAG-based scientific workflow engine — so the same workflow can run its tasks in Docker, Kubernetes (k3s/k0s), Nomad or Apptainer, locally or on remote nodes, and then measured what each choice costs in time, resources and energy.
02Context
Bachelor’s thesis in Computer Engineering at Universidad Carlos III de Madrid (2025–2026), supervised by Dante Domizzi Sánchez Gallegos and awarded 9.5/10. The testbed combined a Fog node (desktop PC acting as orchestrator), an Edge node (Raspberry Pi 5 reading a DHT11 temperature/humidity sensor through an Arduino) and a Cloud tier (MongoDB Atlas).
03Problem statement
Distributed workflows across Edge, Fog and Cloud face three problems — dependency isolation, portability across heterogeneous hardware (x86_64 and ARMv8), and moving intermediate data between nodes that share no filesystem. Choosing a container runtime for constrained edge devices also has measurable consequences for latency, energy and hardware temperature that are rarely quantified.
04My responsibilities
- Designed and implemented task classes for Apptainer, Docker, Kubernetes and Nomad, each with a remote variant, behind DAGonStar’s homogeneous task interface.
- Built a cross-runtime staging mechanism (Stager and Data Mover) combining host-shared areas, SSH/SCP transfers and pod-to-pod streaming for Kubernetes.
- Integrated Prometheus, Node Exporter and Grafana to monitor execution time, resource usage, temperature and energy.
- Designed and ran single-node, distributed and full IoT-telemetry experiments, and analysed the results.
05Architecture overview
The Fog node runs DAGonStar and orchestrates tasks on the Edge node over SSH. On the Edge, the selected runtime — Docker, Apptainer, Nomad or k3s/k0s — executes the tasks that capture and process sensor data. The Stager/Data Mover moves intermediate files between tasks and nodes, results are persisted in MongoDB Atlas, and Node Exporter metrics are scraped by Prometheus and visualised in Grafana.
- Delivery & automation
- Code & configuration
- Platform / runtime
- Device
- Data & storage
- Observability
Text description of the diagram
A Fog node runs the DAGonStar workflow engine and its Stager/Data Mover. It dispatches remote tasks over SSH to a Raspberry Pi 5 edge node running Docker, Apptainer, Nomad or k3s/k0s, with a DHT11 sensor attached via Arduino. Results are persisted to MongoDB Atlas in the cloud. Node Exporter metrics are scraped by Prometheus and visualised in Grafana.
Flows
- DAGonStar → Docker (SSH)
- Stager · Data Mover → Edge — Raspberry Pi 5 (staging)
- DHT11 sensor via Arduino → Nomad
- Apptainer → MongoDB Atlas (persist)
- Edge — Raspberry Pi 5 → Node Exporter (host metrics)
- Node Exporter → Prometheus (scrape)
- Prometheus → Grafana
Components
DAGonStar · Stager · Data Mover · Docker · Apptainer · Nomad · k3s · k0s · DHT11 sensor via Arduino · MongoDB Atlas · Node Exporter · Prometheus · Grafana
06Technology stack
- Runtimes & orchestrators
- Docker
- Apptainer
- HashiCorp Nomad
- k3s
- k0s
- Workflow & data
- DAGonStar
- Python
- SSH / SCP
- MongoDB Atlas
- Observability
- Prometheus
- Node Exporter
- Grafana
- Matplotlib
- Hardware
- Raspberry Pi 5
- Arduino + DHT11
- x86_64 desktop
07Implementation approach
- A class hierarchy of task types — ApptainerTask, DockerTask, KubernetesTask, NomadTask and their remote variants — so users switch runtime without changing how the workflow is written.
- Credentials and connection parameters centralised in a configuration file (dagon.ini) rather than embedded in workflow code.
- Volume-based staging for Docker, Apptainer and Nomad; streaming between pods for Kubernetes, removing the assumption of a shared filesystem.
- Benchmarks implemented as DAGonStar workflows, from edge-only runs to distributed Edge–Fog execution and a complete Edge–Fog–Cloud IoT telemetry workflow.
- Energy, CPU, RAM and temperature monitored in real time with Grafana dashboards per runtime, and post-processed with Python and Matplotlib.
08Challenges & engineering decisions
Challenge
No shared filesystem between Edge, Fog and Cloud nodes.
Decision
A transversal staging layer that picks the mechanism per runtime — shared host volumes, SSH/SCP, or pod-to-pod streaming.
Challenge
Comparing runtimes fairly when they processed different amounts of work (Nomad handled ~150 records, Docker and Apptainer ~292).
Decision
Normalise energy by workload (joules per record) instead of comparing raw energy totals.
Challenge
Kubernetes (k3s/k0s) on a Raspberry Pi 5 with an 8 GB SD card became unstable during long runs as cluster components and images filled the disk.
Decision
Keep the KubernetesTask implementation (fully functional) but report the hardware limitation as a finding, and support lighter runtimes or delegation to the Fog tier.
09Security considerations
- Remote execution and data transfers run over SSH/SCP rather than unauthenticated channels.
- Connection credentials kept in a dedicated configuration file, separate from workflow definitions.
- Identified as future work — end-to-end encryption in staging and secret vaults with automatic rotation for regulated data.
10Operational considerations
- Thermal impact matters on the edge — Nomad’s higher CPU use (10.8% mean vs 0.38% for Apptainer) raised operating temperature by about 5 °C, a throttling risk in long-running or hot deployments.
- Apptainer’s efficiency depends on pre-built images and active storage management; Docker and Nomad handle image distribution more transparently at a higher baseline cost.
11Results & impact
- Apptainer was the most efficient runtime on the edge node — 1.68 J per record, with under 1% orchestration overhead.
- Apptainer reduced energy consumption by 26% compared with Docker and 43% compared with Nomad.
- Docker offered a balanced profile for heterogeneous Edge–Fog deployments; Nomad suits centralised multi-node orchestration but not low-latency, energy-constrained edge workloads.
- Kubernetes executors are functional, but k3s/k0s proved unviable on minimal edge hardware — a documented finding with future work on multi-Pi or HPC clusters.
| Runtime | Records | Energy (J) | J / record | Temp. (°C) | Overhead |
|---|---|---|---|---|---|
| Apptainer | 292 | 490.8 | 1.68 | 46.1 | < 1% |
| Docker | 291 | 666.3 | 2.29 | 47.3 | ~ 29.2% |
| Nomad | 150 | 859.3 | 5.73 | 51.1 | 54.4% |
Source — thesis, Table 4.15.
12Lessons learned
- Normalise before you compare — raw totals hid that Nomad spent most of its energy on orchestration rather than useful work.
- Runtime choice is an infrastructure decision with energy, thermal and hardware-lifetime consequences, not only a developer preference.
- Heterogeneous infrastructure needs an engine that can pick the lightest viable runtime per node, or delegate work upstream.