ML-Affairs Blog
Production notes on ML systems, streaming architectures, and the gap between research and reality
This blog is about the part of machine learning that tends to get skipped in polished case studies: getting systems to behave reliably once they hit production.
I write about streaming-first architectures, event-driven inference, schema and data discipline, observability, and the decisions required to move from “the model works” to “the system survives.”
If you are interested in ML systems that have to run continuously, degrade gracefully, and support real operational decisions, start here.
ML Engineering Needs A Taxonomy
Why ML Engineer, Data Scientist, Data Engineer, and Software Engineer are still used too ambiguously, why that damages hiring, ownership, and delivery quality, and why the industry needs clearer role definitions.
Why read
Read this if you are tired of job descriptions that quietly ask one person to do modelling, pipelines, serving, product thinking, and software engineering all at once.
Who it's for
Especially useful for ML engineers, data scientists, tech leads, and hiring managers trying to build fairer teams and clearer role boundaries.
What you'll learn
Why the term ML Engineer is still too ambiguous, how it overlaps with adjacent roles, and why taxonomy is a coordination tool for hiring, ownership, and delivery quality.
Takeaways
Ambiguous role labels create unicorn expectations and hide real staffing gaps · T-shaped growth is valuable, but expecting full overlap across four disciplines is organisational wishful thinking · The industry needs clearer taxonomy so teams can design interfaces instead of hiring fantasies
Coding Got Cheap. Verification Did Not.
Why AI coding tools increase code supply faster than teams can verify it, and why smaller PRs, merge queues, property-based tests, static analysis, and explicit guarantees matter more than hype.
Why read
Read this if your team is using LLM coding tools and is discovering that faster code generation does not automatically reduce review friction or speed up delivery.
Who it's for
Especially useful for tech leads, staff engineers, and platform teams trying to scale delivery while review, integration, and trust remain stubbornly human bottlenecks.
What you'll learn
Why the real constraint is now verification, why agents still do not carry risk ownership or real agency, and why smaller PRs, merge queues, and stronger guarantees matter more than raw output.
Takeaways
Write throughput is rising faster than verification throughput · Agents can generate code, but they do not own production risk · The practical answer is smaller PRs, better guarantees, stronger automated checks, and disciplined integration
Kafka Streams vs Flink Is The Wrong Question
A production-minded comparison of Kafka Streams and Flink that focuses on state, recovery, rescaling, and platform boundaries.
Why read
Read this if you are tempted to replace Kafka Streams with Flink mostly because Flink feels cleaner, more modern, or more familiar.
Who it's for
Especially useful for streaming engineers and platform teams inheriting messy Kafka-native systems and deciding whether to rewrite or rehabilitate them.
What you'll learn
Where Flink genuinely pulls away, where Kafka Streams is stronger than its reputation, and why familiarity bias is a dangerous architecture strategy.
Takeaways
Flink still wins on runtime flexibility and recovery · Kafka Streams gets very far when the system is still application-shaped and Kafka-native · Rewriting because a framework feels old is often familiarity pretending to be engineering judgment
PyFlink In 2026: Better Than Its Reputation, Still Not Frictionless
Why PyFlink becomes attractive once Python training and Java prediction start drifting apart, and where the JVM/runtime boundary still costs you.
Why read
Read this if your team trains in Python, predicts in Java, and keeps paying for that split in latency, feature drift, and debugging time.
Who it's for
Boldly useful for ML platform engineers and streaming teams forced to split training, feature logic, and inference across Python and JVM services.
What you'll learn
Why Python-native feature and model logic is the real case for PyFlink, where ONNX and model-as-a-service help, and where the runtime still pushes back.
Takeaways
Cross-language feature pipelines create hidden production bugs · Model-as-a-service buys separation but adds latency and dependency costs · PyFlink is strongest when same-ecosystem logic matters more than JVM purity
From Model Validation To Pipeline Validation
Why retrospective ML evaluation needs pipeline validation, not just model validation, and how future leakage can distort backtests.
Why read
Read this if your team evaluates historical ML predictions and wants to avoid quietly leaking future information into past decisions.
Who it's for
Especially useful for ML engineers, data scientists, and platform teams building retraining, backtesting, or retrospective validation workflows.
What you'll learn
Why one impressive model is not enough, how the Future Leakage Paradox distorts retrospective evaluation, and why the training pipeline itself has to become the validation target.
Takeaways
Retrospective validation breaks when the model has already learned the future · Sliding and expanding windows test the pipeline, not just one model · Traceability, determinism, and temporal stability are production requirements, not nice-to-haves
Harmonizing Avro and Python: A Dance of Data Classes
Why Avro schema discipline matters in data engineering and how to generate Python data classes from Avro schemas.
Why read
Useful if you want a practical reason to care about Avro-level schema discipline instead of treating it as ceremony.
Who it's for
Data and ML engineers who need stricter schema discipline without giving up Python-friendly developer workflows.
What you'll learn
Why Avro matters, where schema drift hurts, and how to turn Avro definitions into Python-friendly data classes.
Takeaways
Avro schema management prevents downstream breakage · Serialization choices shape system reliability · Python data classes can fit a stricter data contract
Agile In Action: Bridging Data Science and Engineering
What Agile looked like to me in 2023 at Vortexa: helping data scientists and engineers learn together, communicate clearly, and ship ML systems that can survive production.
Why read
Read this if your ML work keeps getting stuck between promising notebooks and production systems that nobody fully owns.
Who it's for
Especially useful for data scientists, software engineers, and ML leads trying to make cross-functional teams work without hiding behind process.
What you'll learn
Why Agile mattered to me less as ritual and more as a way to synchronise experimentation, communication, and engineering discipline.
Takeaways
Agile is useful when it makes experiments explicit · The real gap sits between a notebook and a production system · Communication is part of the system, not an afterthought
Dynamic(i/o) Why you should start your ML-Ops journey with wrapping your I/O
Why ML pipelines need I/O abstraction, and how wrapping file and storage concerns improves maintainability, testing, and delivery.
Why read
Read this if your ML codebase keeps getting buried under storage, schema, and environment-specific glue code.
Who it's for
ML engineers and platform builders whose pipeline code is dominated by file, storage, and schema handling instead of core logic.
What you'll learn
Why I/O concerns overwhelm ML pipeline code, and why abstraction helps more than another pile of helpers.
Takeaways
I/O logic should not dominate business logic · Abstraction improves testing and portability · Cleaner boundaries make ML systems easier to maintain
Complete Guide to Python Envs (MacOS)
A practical guide to Python environments on macOS, including system Python pitfalls and the setup patterns that actually work.
Why read
A practical post for anyone who is still losing time to Python setup confusion on macOS.
Who it's for
macOS users and Python developers who are tired of not knowing which interpreter or environment they are actually using.
What you'll learn
Which Python environment tools matter, where macOS gets in the way, and how to build a sane local setup.
Takeaways
System Python is rarely what you want · Environment management is a productivity issue · Clarity beats collecting tools
A BREXIT NLP Dataset!
How I built a BREXIT-related NLP dataset and why real-world labeling and collection choices matter in applied NLP.
Why read
Useful if you care about how topical NLP datasets get assembled outside clean benchmark settings.
Who it's for
Practitioners building applied NLP datasets from messy real-world sources rather than benchmark-ready corpora.
What you'll learn
How the dataset idea emerged, why labeling is the hard part, and what makes a practical NLP dataset usable.
Takeaways
Real data collection is messy · Labeling strategy determines downstream value · Applied NLP starts with better datasets, not just models
Style Transfer in Heraklion
An intuitive explanation of neural style transfer, grounded in Heraklion imagery and the core mechanics behind the model.
Why read
A more intuitive route into neural style transfer than a dry paper summary.
Who it's for
ML practitioners and curious readers who want to understand style transfer through intuition before diving into the paper.
What you'll learn
The core mental model behind style transfer, how the losses work, and why the generated image changes.
Takeaways
Style and content are separated mathematically · The optimization target is the generated image · A visual example helps the theory land
Agile Data Science
Why agile matters in data science work, where experimentation meets delivery, and how teams can move from prototypes to value.
Why read
Read this if you want a more grounded view of what Agile can actually mean in data science work.
Who it's for
Data scientists and ML engineers trying to move from prototype work into production delivery without losing iteration speed.
What you'll learn
Where Agile helps, where data science work differs from normal software delivery, and why iteration matters.
Takeaways
Experimentation needs structure · Delivery and learning loops must coexist · Agile in ML is about adaptation, not rituals
AWS ML Certification
A practical guide to the AWS Machine Learning Specialty exam, including difficulty, study scope, and how long preparation usually takes.
Why read
Useful if you are deciding whether the AWS ML Specialty is worth the effort and how to prepare for it.
Who it's for
Data scientists, ML engineers, and cloud practitioners deciding whether to invest in the AWS ML Specialty exam.
What you'll learn
How hard the exam really is, how its topics are split, and what kind of study timeline makes sense.
Takeaways
The exam is broad, not just AWS services · Preparation time depends heavily on your background · A structured study plan matters more than cramming
Just do it!
A short reflection on why public writing matters, why criticism is not the point, and why you should publish before you feel ready.
Why read
A short, honest reminder that writing in public matters more than waiting to feel fully ready.
Who it's for
Engineers, builders, and first-time writers who keep delaying public writing until it feels safe.
What you'll learn
Why blogging feels exposing, why that fear is normal, and why publishing imperfect work still has real value.
Takeaways
Write before confidence arrives · Criticism is part of the exchange · Blogging is practice, not performance