A "hello world" to machine learning, and solving a real-world inefficiency? Sign me up! This article will share some of the highlights and "Ah-ha!" moments during the build of my latest React app - Color Correct.

Some Context...

Print has always had fairly thin profit margins, and as a result, innovation and lean manufacturing is the name of the game. This can typically be observed at all levels of a print business. I have spent a fair amount of time on the front lines of production in a few print/packaging environments and have found my share of ways to improve efficiency.

The goal of this project is to reduce paper waste, ink waste, and labor hours on a modern printing press. A quick outline of some important print industry terms before moving forward:

• Make-ready - This can refer to the process of setting a press up to run a job, or more specifically, in this case, the sacrificial sheets before "good" sheets start coming out of the machine.
• Print Unit/Print Tower - One "section" of the press responsible for transferring a single color to the sheet.
• Ink Fountain - Each Print Unit's ink resevoir, ink is transferred from here directly to the roller-train
• Ink Key - There are a series of "flaps" that meter the amount of ink let through to the roller train. These allow the fine-tuning of ink density per region

The goal

In lithographic printing, there is a "prepress" department that is responsible for job layouts, creating print plates. Along with the print plates, prepress will generate a file to send to the press that tells the press how much ink to lay down in each area based on how heavy the coverage is in the image. This file is seen as a starting point for the press, and the pressman will adjust from here, and do fine-tuning on the fly. In a lot of cases, these files are not spot-on, due to mechanical changes on the press, ink/water balance, using different chemistry, impression, any one of a seemingly endless list of variables. My goal is to adjust the files from prepress to make for shorter make-readies and more accurate color densities early in each print run. This should reduce paper waste, ink used, and time spent on make-readies.

The Approach

The files prepress generates start as something with a proprietary encoding, however, the final file sent to the press is human-readable. And a nearly identical format is exported from the press when saving a color profile... We can use these exports to compare against the originals and treat this is a polynomial regression problem. I have not done a whole lot with neural nets or machine learning in general, and this seems like a great problem to solve with a neural net. I'll extract the ink key data, convert to tensors, and use TensorFlow to establish the relationship between the two - find the "curve" so to speak.

Each row of the prepress files corresponds to one print unit on the press. Each value/column in the row, corresponds to one of the 34 ink keys across each unit. This will make rank-2 tensors that are easy to work with. I know a few things going into this, that stand to create hurdles in the process:

1. I will have small datasets (ideally I can train this model on less than 10 samples)
2. I need to localize the influence the ink keys have on each other; I do not want an ink key from one side of a unit to influence the output of a key at the opposite end.

Neural Net Architecture

I want each ink key to take some amount of influence from it's immediate neighbors, but not further away than that. Given the small datasets and the goal of performing regression on smaller subsets of a larger data array, I will need to be creative here. There may be a few ways to separate each key into it's own zone, but the easiest solution seems to be by using a custom (non-trainable) weight matrix to define my zones, and perform the actual regression in the following layer. There are 8 print units, each with 34 keys across, this is 272 individual keys. As such, I will flatten my data and the input will be a vector of 272 ink key values. The architecture I've settled into is illustrated below.

Following the input, the first layer is actually 272 parallel single-node layers, they have a fixed weight of 1 for the corresponding node in the input layer, and a fixed .35 for the immediate neighbors. Moving forward these continue as parallel layers, though are allowed to train and develop bias. Both of these layers use linear activation functions, as it is never a question of should a neuron fire or not - they all need to fire. The next layer concatenates these back to (1,272), and not imaged here - reshapes to (8,34), to resemble the original (print units) X (ink keys).

The frontend (for now)

Embedded below is a very quick demo showing the current functionality. The frontend uses react, and react-redux for state management. (there is no audio, don't bother turning your sound up)

Optimizing user interaction

This takes the form of a web app to avoid needing permissions to install or run on a work network. For now the workflow is a bit archaic, but functional.

In the meantime, we will continue sporting drag n' drop elements, and a bare-bones database to save models. The TensorflowJS library generates a multipart/form-data MIME type request with the model architecture as JSON and weights as a binary. This should allow for importing model and weights to Tensorflow's other language libraries... I sense an API coming soon.

Work in Progress

Unlike my graveyard of half projects, this one is not currently on the back burner. Right now I am in somewhat of a testing phase, sorting out when during a print run is best to export ink settings to train with. Every printing press has its own "personality", and while I am getting results that make a lot of sense from the model, honing in on the right training data will really make this shine.

Update 12-14-24

This did, in fact, find it's way to the back burner, and finally the graveyard.
I have since moved away from the print industry and no longer had a need to improve this. However, should anyone come across this that can make use of this, please reach out (danieljhalleck@gmail.com) - I can host or would be happy to collaborate or help integrate this tech to see it used in the real world.

The majority of this last year has been light on programming. That said, I found an immediate need for a quick text generator to recreate the "100 points" emoji with [mostly troll] text.

A handful of the technologies I will use listed below, spoiler alert (it's just what I am most comfortable with):

• React (Vite)
• Bootstrap (for easy responsiveness)
• GO (for any backend needed)
• MySQL (found a excuse to implement a database)

The Foundation

First thing's first, I need a font. After comparing a handful of these "100" emojis, I've now seen many different fonts, and slightly different styles. Google fonts was a first and last stop to find a font with a touch of handwriting style, without being too Comic Sans-y. Shantell Sans SemiBold 600 Italic? Sold!

Import a font, apply that to a canvas, write strings to canvas, draw some lines under that. The following is the logic portion of the React component that produces the emoji.

	//Excuse the formatting

	const [currentText, setCurrentText] = useState("");

	const canvas = document.getElementById("canvas");
        const ctx = canvas.getContext("2d");



            //background
            ctx.canvas.width = ctx.measureText(currentText).width + 20
            // ctx.fillStyle = "black";
            ctx.globalAlpha = 0;
            ctx.fillRect(0, 0, canvas.width, canvas.height);


            //text
            ctx.globalAlpha = 1;
            ctx.font = "50px myFont, myFontCSS";
            ctx.fillStyle = "red";
            ctx.fillText(currentText, 10, 50);
            const firstArcStart = 1.4815;
            const secondArcStart = 1.4822;
            const radian = ctx.measureText(currentText).width / ((2 * Math.PI) * 3690) * (2 * Math.PI);


            //line1 
            ctx.beginPath();
            ctx.strokeStyle = "red";
            ctx.lineWidth = 6;
            ctx.lineCap = "round";
            if (currentText.length > 0) {

                ctx.arc(220, 3606, 3550, firstArcStart * Math.PI, 		   (firstArcStart * Math.PI) + radian);
            }
            ctx.stroke();

            //line2
            ctx.beginPath();
            ctx.strokeStyle = "red";
            ctx.lineWidth = 6;

            //Below some width, the bottom arc will "wrap". This conditional is to not use an "offset" when text length is below 21
            if (ctx.measureText(currentText).width > 21) {

                ctx.arc(220, 3618, 3550, secondArcStart * Math.PI, ((secondArcStart - .0014) * Math.PI) + radian);
            } else if ((ctx.measureText(currentText).width > 0) && (ctx.measureText(currentText).width <= 21)) {
                ctx.arc(220, 3618, 3550, secondArcStart * Math.PI, ((secondArcStart) * Math.PI));
            }
            ctx.stroke();

This logic is actually in a function of its own, but for the sake of context is shown here with state variable and selectors. Surely the circle-math needs some refactoring, but it works well, as is for now, so it stays.

From here, I did wrap the logic in a conditional to handle when the measure of the string in pixels exceeds a certain length. The has a large enough radius to cover a good span, but short enough to show the "swoop" on single character. A dynamically changing radius is outside the scope of this project, and static/straight lines are not desired - also not interested in creating a font.

Copy-ability

It became apparent very quickly that, while using most browsers on PC, you can just right-click, copy the canvas, and the browser will handle turning this into a PNG. Safari on iPhone was the first indication that this was not going to work.

Easy fix - I use the canvas method canvas.toDataURL(), which returns the canvas as a base64 string and it's respective MIME type. I'm now using this as the src for <img> elements and hiding the canvas that produced it.

Most Recent Emojis Rendered

I needed some excuse to start up a database and backend. What better way than a feature that displays the last 5 strings entered. For this, I am using Gin in Go to serve up the world's smallest API. Of course this could have been done entirely within the standard library, but Gin just has so many GitHub stars and I like learning more about tools that are used at scale.

The strings are stored in a database on the same server. The backend is hosted on a port other than :80 or :443 and uses NGINX's reverse proxy feature to route requests accordingly.

/strings (submits current string)
    POST accepts: {string: <string>}
/lastfive (returns last five [unique] strings entered)
    GET returns: Array(5) {time: <string>, phrase: <string>}
    [
    {
        "time": "2024-01-17 15:19:02",
        "phrase": "Thisisalongstring"
    },
    ....]

//TODO create endpoint to submit a string and return the DataURL representation/png file by API call.

When first visiting the site, there is a useEffect hook with an empty dependency array to fetch from this endpoint. This will be iterated over using our function from earlier, and return a handful of <img> elements and a relative time description (currently using momentJS).

Summary

This is my tentative summary.. It has been a good project so far to get back in the groove. It is still very basic - nearly no styling. Error handling needs some real attention. It has been fun so far.

Although it serves no real utility, I'm happy to have introduced something to the internet that I have not seen yet. I plan to pick at this moving forward in my free time.

A very creative, SEO friendly domain: 100emojigenerator.com

Cache me outside, how bout dat

While this is a very specific search term and target, it is still neat to see one of my sites show up as a first result.

I will need a GUI of sorts for the user to begin using bluetooth input for something.

Wails allows you to build a desktop app using modern frontend web technologies including, but not limited to React. Wails also uses Vite under the hood, which is exciting for someone who has historically used create-react-app. It hardly matters to me in this case if somewhere far abstracted from me is: webpack, rollup, or esbuild, but using a cutting edge tool like this is too exciting to pass up.

Well, it's not pretty (yet), and it's not robust (yet), but we've the beginnings of a GUI to work with. To this point, I've used the Wails tool to spin up a Vite-React project, and for no particular reason I am styling using Tailwind. Ultimately, I'll just need a few neat tables, a diagram or two, and a menu of sorts. Using the Bluetooth Assigned Numbers documentation, I should be able to infer the names of the non-custom services and characteristics as well.

I've opted for a Vite-React project, so the frontend updates are lightning fast. When updating Go files, it does have to perform a full rebuild as it creates JS bindings for your Go functions and builds a new binary. The creator of this tool seems reasonably dedicated to keeping the single file binary dream alive, and from what I've seen, v3 will not disappoint.

The Go functions are called in JS exactly as they are "exported". In JS, the Go function's return shows up as a promise - easy enough. I've opted to wrap all of my Go functions on the JS side so I can add some JS logic and clean up the bigger files.

An interesting roadblock..

The official Bluetooth documentation is written well enough.. however, there are so many different types of data, it quickly becomes clear this would require an unreasonable amount of time and effort to support all devices. In the current build, I have borrowed the bluetooth-numbers-database data from Nordic Semiconductor's github. I have this list hard-coded in my project as follows, and it is great for displaying service/characteristic names of common devices.

This proves only so useful, as each and every characteristic has a unique number of bits it sends, representing something specific, as defined in the bluetooth documents. While I plan to allow users to perceive the data in their own way and map it accordingly, there must still be a solid foundation within the software on which to provide the user some options.

As much as I would love to map the frequency at which you ignore Jeff Bezos' phone calls to some mechanic in a game, it just is not going to happen, not dynamically anyway.

The solution

--This project currently on the back-burner. Left off in the early stages of writing the Windows driver--

The goal here is to take input data from sensors attached to a bicycle, and build a no-frills GUI to map this data to a virtual controller. The user should ultimately be able to map this data to the functions of a common game controller with a level of precision that allows for a good gaming experience.

I'd like to use as many sensors that already exist and are on the market, so to get started, I will focus on a Wahoo sensors for both wheel speed and pedal cadence. Steering, braking, and everything else will be a bit more custom.

How does bluetooth work anyway?

I've chosen to stick with Bluetooth Low Energy instead of the ANT+ protocol, for both quicker transfer speeds and a [slightly] lower barrier to entry. Let's start with a Wahoo wheel speed sensor.

I will be using Bettercap to learn a little more about this sensor and it's BLE packet information.

To make better sense of what is going on here, I will refer to the official BluetoothSIG Assigned Numbers documentation. BLE uses 16-bit UUIDs to label services and characteristics, with their docs we can verify we have:

1816 - Cycling Speed and Cadence [service]

and within that service we have:

2a5b - CSC Measurement [characteristic]

So far I have only made a single connection to the device to obtain some base information, to expand on this I will need to "subscribe" to the characteristics I want to stream data from. This characteristic has a property called NOTIFY which means the peripheral device will continuously send updates of this characteristic to it's subscribers. The READ properties make characteristics useful for initial setup but have no logical reason to stream as they should not change.

2a5c - CSC Feature [characterstic]

Our feature field reads '0100'. Following the chart, we know there are going to be up to 16 boolean values to make up our 2 octets. At first glance this might suggest we have only Crank Revolution Data Supported, but this is a Wheel Speed Sensor. Using the Nordic Semiconductor app for android, it is properly interpreted as wheel data, making our output from bettercap all the more confusing. We'll get back to this.

I will be using TinyGo's bluetooth package, certainly there are other methods, but for this project I will be introducing myself to Wails later and would prefer to keep the software itself Go-heavy. The code is omitted for now as we will be very much in-the-weeds on that later.

The introduction portion of the Bluetooth docs contain two very important lines that are essential to interpreting this data:

TinyGo delivers the data to us in uint8 values, the gif above shows us an array of said values representing the entire CSC measurement characteristic. Referring to the CSC Measurement figure above we can ascertain the following:

The "Last wheel event time" unit of measurement is 1/1024th second as per the Bluetooth GATT docs. Notice the rollover on this metric, we also learned in the docs that this is represented by a uint16 number.

Summary

This has been a great intro to bluetooth data, surely building on this foundation will be imperative to moving forward on the larger project at hand. I now have some useful data here, I can actually calculate a speed with some basic math and bike measurements. In combination with a crank speed sensor I will be able to estimate power output from the rider and gearing. For my application that may be irrelevant, but I'll take all the data I can get! While the data appears to come in just over once per second, the "last wheel event time" field is going to help smooth the data in practice - a very interesting way of dealing with uncertain transmission.