Valohai logo
Blog / Juha Kiili
Juha Kiili

Juha Kiili

Senior Software Developer with gaming industry background shape-shifted into a full-stack ninja. I have the biggest monitor.

Blog
August 23, 2022 Juha Kiili
Modern web scraping pipeline for ML

Web scraping and data gathering are vast topics. There is no single correct way to programmatically collect data from sources designed for human consumption. The right approach for web scraping depends on the context, and in this article, we focus on an early-stage ML project needing time series data.

View post
Blog
August 17, 2022 Juha Kiili
Continuous training and deployment for machine learning at the edge

Michael Vakulenko of JFrog and Juha Kiili of Valohai showcase the real-life example of creating a machine learning system that continuously improves itself through IoT.

View post
Blog
July 25, 2022 Juha Kiili
Makefile: the secret weapon for ML project management

What if I told you there is a simple, free, lightweight tool for weaponizing any CLI-based ML project. All the commands are nicely wrapped and accessible via shortcut aliases and only a TAB keypress away. This tool is easily installable and super robust for all operating systems! It is called Make.

View post
Blog
July 04, 2022 Juha Kiili
Five things to know about Jupyter notebooks

Here are some tips and tricks for Jupyter notebook with step-by-step guides: from running shell commands to changing the notebook theme - easily!

View post
Blog
June 20, 2022 Juha Kiili
IDEs for Data Science: What every data scientist should know about programming tools

There is absolutely nothing wrong with notebooks, and they are fantastic for many use-cases, but they are not the only option for writing programs. Too many get stuck in the vanilla notebook and do not realize what they are missing out on.

View post
Blog
May 16, 2022 Juha Kiili
What every data scientist should know about the command line

Almost any programming language in the world is more powerful than the command line. Why would you even bother doing anything on it? Don't be fooled: the modern command line is rocking like never before!

View post
Blog
March 10, 2022 Juha Kiili
Experimentation at Scale: a Q&A with Serg Masís from Syngenta

Syngenta is a leading provider of agricultural science and technology focused on seed and crop protection products aiming to improve global food security by enabling millions of farmers to make better use of available resources.

View post
Blog
February 14, 2022 Juha Kiili
Docker for Data Science: What every data scientist should know about Docker

Docker isolates the software from all other things on the same system. A program running inside a "spacesuit" generally has no idea it is wearing one and is unaffected by anything happening outside.

View post
Blog
January 27, 2022 Juha Kiili
What every data scientist should know about Python dependencies

Dependency management is the act of managing all the external pieces that your project relies on. It has the risk profile of a sewage system. When it works, you don't even know it's there, but when it fails, it becomes very painful and almost impossible to ignore.

View post
Blog
January 04, 2022 Juha Kiili
Git for Data Science: What every data scientist should know about Git

Git is a tool most software developers have used daily for a decade, and with data scientists becoming an integral part of R&D teams, Git is every day for them as well. We've listed a few helpful tips on using Git for your ML work and avoiding the common pitfalls.

View post
Blog
November 24, 2021 Juha Kiili
Product Update: Human Validation and Confusion Matrices

We’ve recently introduced two features that make building trusted and validated models easier: human validation steps and confusion matrices.

View post
Blog
October 11, 2021 Juha Kiili
Product Update: Debugging and Metadata

For the October product update, we chose to highlight a new feature, Remote Access Debugger, and some major improvements that we've shipped to the Metadata View.

View post
Blog
August 31, 2021 Juha Kiili
Product Update: Spark as a First-Class Citizen

Support for Spark has been one of the most requested features as Spark has become almost ubiquitous for data scientists and engineers working with structured data. We’ve heard the calls and Valohai now supports Spark natively.

View post
Blog
July 07, 2021 Juha Kiili
Product Update: Datum Improvements

Datum is a version-controlled file inside the Valohai platform. Every datum is immutable by design. We have introduced three new improvements for more flexibility over datums.

View post
Blog
June 08, 2021 Juha Kiili
Building a YOLOv3 pipeline with Valohai and Superb AI

This article shows an example of a pipeline that integrates Valohai and Superb AI to train a computer vision model using pre-trained weights and transfer learning. For the model, we are using YOLOv3, which is built for real-time object detection.

View post
Blog
May 31, 2021 Juha Kiili
Product Update: Kubernetes, Spot Instances & Python Utility Library

It's time for an update on what's been happening under the hood of the Valohai platform. We'd like to highlight three major features we've added in the past two months: Support for Kubernetes and Spot instances and the Valohai Python utility library.

View post
Blog
March 31, 2021 Juha Kiili
Superb Meets Valohai: An End-to-End Solution for Developing Computer Vision Applications

Computer vision is one of the most disruptive technologies of the recent decade. To develop computer vision systems requires massive, upfront investments. Or it used to, before Superb met Valohai.

View post
Blog
February 03, 2021 Juha Kiili
Product Update: End-to-End Automation

In the past few months, we've rolled out three new features that highlight end-to-end automation on our platform: Deployment nodes in pipelines, Pipeline scheduler & Model monitoring.

View post
Blog
December 15, 2020 Juha Kiili
How We Trained 277M Models for the Black-Box Optimization Challenge

Valohai MLOps platform provided the infrastructure for the Black-Box Optimization Challenge for the NeurIPS 2020 conference. The competition was organized together with Twitter, Facebook, SigOpt, ChaLearn, and 4paradigm.

View post
Blog
October 31, 2019 Juha Kiili
Updates for Valohai Powered Notebooks

Valohai is the enterprise-grade machine learning platform for data scientists that build custom models by hand. In addition to writing code with classic IDEs like PyCharm or VSCode, we also have native support for data scientists preferring to use Jupyter notebooks.

View post
Blog
September 10, 2019 Juha Kiili
Self-Driving with Valohai: How to Train an Autonomous Driving Model Using Deep Learning

One of the hottest areas of application for deep learning is undoubtedly self-driving cars. We’ll go through the problem space, discuss its intricacies and build a self-driving solution utilizing the Unity game engine, training a neural network on top of the Valohai platform. Regardless of the technologies used, you’ll get an understanding of the basics as well as the code to tweak for yourself.

View post
Blog
May 28, 2019 Juha Kiili
Valohai's Jupyter Notebook Extension

Valohai is a deep learning platform that helps you execute on-demand experiments in the cloud with full version control. Jupyter Notebook is a popular IDE for the data scientist. It is especially suited for early data exploration and prototyping.

View post
Blog
May 28, 2019 Juha Kiili
Asynchronous Workflows in Data Science

Pointlessly staring at live logs and waiting for a miracle to happen is a huge time sink for data scientists everywhere. Instead, one should strive for an asynchronous workflow. In this article, we define asynchronous workflows, figure out some of the obstacles and finally guide you to a next article to look at a real-life example in action in Jupyter Notebooks.

View post
Blog
May 02, 2019 Juha Kiili
From Zero to Hero with Valohai CLI, Part 2

Valohai executions can be triggered directly from the CLI and let you roll up your sleeves and fine-tune your options a bit more hands-on than our web-based UI. In part one, I showed you how to install and get started with Valohai’s command-line interface (CLI). Now, it’s time to take a deeper dive and power up with features that’ll take your daily productivity to new heights.

View post
Blog
April 04, 2019 Juha Kiili
From Zero to Hero with Valohai CLI, Part 1

As new Valohai users get acquainted with the platform, many fall in love our web-based UI - and for good reason. Its responsive, intuitive and gets the job done with just a few clicks. But don’t be fooled into thinking that’s the end of the interface conversation. We know it takes different [key]strokes for different folks, so Valohai also includes a command-line interface (CLI) and the REST API.

View post
Blog
March 27, 2019 Juha Kiili
TensorBoard + Valohai Tutorial

One of the core design paradigms of Valohai is technology agnosticism. Building on top of the file system and in our case Docker means that we support running very different kinds of applications, scripts, languages and frameworks on top of Valohai. This means most systems are Valohai-ready because of these common abstractions. The same is true for TensorBoard as well.

View post
Blog
March 13, 2019 Juha Kiili
Automatic Version Control Meets Jupyter Notebooks

Running a local notebook is great for early data exploration and model tinkering, there’s no doubt about it. But eventually you’ll outgrow it and want to scale up and train the model in the cloud with easy parallel executions, full version control and robust deployment. (Letting you reproduce your experiments and share them with team members at any time.)

View post
Blog
February 27, 2019 Juha Kiili
Reinforcement Learning Tutorial Part 3: Basic Deep Q-Learning

In this third part, we will move our Q-learning approach from a Q-table to a deep neural net.

View post
Blog
February 07, 2019 Juha Kiili
Reinforcement Learning Tutorial Part 2: Cloud Q-learning

In this second part takes these examples, turns them into Python code and trains them in the cloud, using the Valohai deep learning management platform.

View post
Blog
January 24, 2019 Juha Kiili
Reinforcement Learning Tutorial Part 1: Q-Learning

This is the first part of a tutorial series about reinforcement learning. We will start with some theory and then move on to more practical things in the next part. During this series, you will not only learn how to train your model, but also what is the best workflow for training it in the cloud with full version control using the Valohai deep learning management platform.

View post
Blog
December 19, 2018 Juha Kiili
Run Jupyter Notebook On Any Cloud Provider

This tutorial will demonstrate how to take a single cell in a local Jupyter Notebook and run it in the cloud, using the Valohai platform and its command-line client (CLI).

View post
Blog
November 21, 2018 Juha Kiili
PocketFlow with Valohai

PocketFlow is an open-source framework from Tencent to automatically compress and optimize deep learning models. Especially edge devices such as mobile phones or IoT devices can be very limited on computing resources so sacrificing a bit of model performance for a much smaller memory footprint and lower computational requirements is a smart tradeoff.

View post