How to Develop and Include ML solutions for Business
Back-end Developer at JetRockets
There is a crazy demand for solutions using ML and generative neural networks. People constantly ask how they can implement AI into their product. Well, this is not surprising. Otherwise, terminators will take over the entire market.
Well, okay, let's move on from jokes to action. Here, I think there are two questions to consider:
- “What does the development process for ML solutions look like?”
- “What kind of team do you need, and what areas of responsibility should there be?”
Why these two questions? Without understanding what the development pipeline will be and what kind of people are required on the team, it is impossible to effectively complete the task. I will note that I will not analyze in detail each point of the development process, since that is information for separate articles. I will only briefly examine the issues to the point.
Business request
If there is no request, then there is no machine-learning task. Accordingly, before someone starts doing something, there must be a customer who arranged a “banquet”. There can be an endless number of options for a business request, so I will try to generalize these points to our topic.
Let's imagine that a customer comes and tells his problem. Let's say this is an application with short videos, but people do not stay long on the service because they do not find what interests them there. We want to know if there is any recommendation system. The customer says no. Then, we decide to evaluate the possibility of its implementation - here is the task of machine learning. If the answer is “Yes, we have a recommendation system,” we have to figure out why it performs poorly and what can be done about it.
At the same time, it is important to note that the customer could decide for some reason that people are leaving because they don’t find what interests them. However, the reason may be unfriendly UI/UX design or something else. Maybe no one uses the application, so there is nothing to recommend. Therefore, you should always be critical of what you are told and ask for some research done by an analyst or analyze the situation yourself.
The customer himself can decide that this cannot be done without ML. Here, too, you should be suspicious of this request since it may turn out that ML algorithms are useless here. For example, the following offer comes up: “We want to display personalized articles in our app. Make us a recommendation algorithm.” But it turns out that there are only 15 articles, which may be divided into only 3 categories, and new articles are written every six months by one of the admins. The question arises: why make a recommendation system? In this case, tags and a convenient menu would work just fine.
The most important thing is not to be afraid that there is eventually no machine learning task in your business request. If the problem can be solved without it, this is even better since you saved people money and time.
Who converts a business request to an ML task?
Let us begin to reveal who is indispensable to the team in solving all these issues. A Data Scientist and/or ML Engineer is responsible for processing a business request. Which one is better to choose? This understanding will come by the end of the text, but for now, let’s figure out who they are and how they differ.
There is an opinion that the skills of all the specialists we will consider come from Data Science expertise. In other words, the original data science, which contains many aspects, gradually began to separate its tasks.
Likewise, an ML engineer is one who takes on the development and design of ML solutions. This is a narrower and more specific developer on the one hand but at the same time multitasking on the other. This sounds ambiguous, but later, you will understand why I say this.
A data scientist is more of a scientist who does a lot of research. I would say that a DS specialist deals with a business request, when the task is to research something, find some trends, i.e. on a more abstract level. In this case, of course, the DS specialist is capable of doing what was assigned to the ML engineer, but in the reverse order, it will probably not always work.
Define a target variable
Let's move on to the next task. A target variable is what you want to predict. Let's return to our service with short videos. It makes sense that, in order to recommend something, you have to understand what the user likes. Consequently, it is important to know what exactly to predict and find the variable that shows whether the user liked something or not.
In general, the same likes are suitable for this. If a user “liked” the video, it means they like it. If they did not “like”, they do not. Naturally, in modern services with short videos, everything is somewhat more complicated: such things as viewing time, shares, and likes are taken into account. Let us focus on one thing to make the example clearer.
Before defining anything, it is worth checking what data we have. Perhaps there is not enough data to start working. Also, most likely, the data will arrive in an inconvenient form and will have to be transformed. Basically, it is necessary to do some analysis before starting to predict something.
After analysis, we can come to the conclusion that it is impossible to implement an ML solution since, for example, the application does not have likes and does not count metrics, so there is nowhere to get data from. Therefore, first we have to highlight what we want and ask the customer to implement the proper solutions. The next step is to wait for the data to accumulate and continue working.
Who defines a target variable?
A data analyst and/or ML engineer does this sort of thing. Here the situation is different, because ideally, these two specialists work together. After all, this is what an analyst specializes in, finding relationships: he does high-quality research and prepares beautiful visualization what options for solving an issue there are and why.
An ML engineer, if he does this, will conduct a quick analysis to solve his problem. It's not something bad. Sometimes this is exactly what is right, but there are cases when it is best if the analysis be carried out by an analyst. If there is no analyst, an ML engineer must be capable of doing A/B tests and monitoring the state of the model after release.
As I already wrote, it would be ideal if they worked together. When the ML engineer indicates what he wants to see, together with the analyst, he performs better.
Transform data
Often, the dataset is constantly changing - data is added to it, sometimes every day. Accordingly, it is necessary to automate the process of processing new data for their further use. Just to clarify, when new users or videos appear on a service with short videos, this is precisely the data that has to be added to the dataset. This is important because the model can work with relevant data.
To automate this, ETL processes are created, i.e., first, the data is extracted, then transformed and loaded. In practice, data always needs pre-processing since, unfortunately, it does not arrive in an ideal state that is suitable for the ML model to work.
Who does data transformation?
At this phase, a new hero appears - the data engineer. This is a specialist who is engaged in uploading and preprocessing data for convenient interaction with them. The result of his work is regularly updated data marts, a database that is designed to perform a specific task.
In fact, a data engineer has a much longer list of competencies in preparing and working with data than an ML engineer. At the same time, it is most often an ML engineer who creates the dataset from ready-made data marts. Although even a data engineer can collect a dataset according to technical specifications if he is competent enough in ML issues. Usually there is no data engineer in the company and all issues related to data transformation are dealt with by the ML engineer.
Ideally, these two specialists work together. Then, the task gets completed faster and better.
Model creation and retraining
I do not want to burden you with unnecessary technical details, so I combined the next two stages, “Creating a model” and “Retraining the model,” into one point.
When everything is fine with the business request, a target variable is defined and data is prepared, you can start creating an ML model.
The main thing is to check ready-made solutions, which are especially desirable to use. There is no point in creating a clone of GPT-4 or Llama 2 if the goal is to create LLM. It is easier to take an existing solution and retrain it in accordance with given tasks. If our alternative is more custom, it is better to select the optimal model architecture, hyperparameters and configure feature filtering.
There will be a lot of testing at this stage to make sure everything works as it should and the model does what is mostly expected. Unfortunately, 100% results cannot be achieved. In some cases, even 80% success is a lot. Therefore, be patient and prepared for the fact that the model will not do its job perfectly.
The next point is that the model needs to be constantly retrained. Data is updated so that the patterns between a target variable and features may change. Consequently, work success decreases. Thus, an automated process of model retraining is required. How often is the model retrained? Depends on the amount of data and how often it is updated. Maybe once a month, once a week or more often.
These issues are dealt with either by a DS specialist or an ML engineer. I wrote about their difference above. If you forgot what it was about, I recommend coming back.
Deploying a model to production
The last, but not least important, step in developing an ML solution is the deployment and use of the model itself.
The model works on the service, which can be accessed to get a response. If an ML solution is being implemented into an existing application, then the service is required to be implemented. In fact, the task is not always trivial, especially if it is some old and large application.
The infrastructure for our service is of primary importance. Moreover, it differs significantly from the infrastructure for conventional applications. For example, if we want to work with a full-fledged, unoptimized Llama 2 and deploy it ourselves, then the server must have at least 48 gigabytes of video memory. Of course, the question is limited not only to hardware, but also to software.
One of the important features in the operation of an ML service is the presence of artifacts, i.e the model and data on which it was trained. All issues in working with artifacts have to be automated, given that the size of artifacts can reach hundreds of gigabytes.
In addition, such services have specific CI/CD (continuous integration and continuous delivery), without which it is impossible to constantly update and deploy them.
Who deploys a model to production?
As you can see, the last point is very difficult. Who will do all this? Ideally, an MLOps engineer is responsible for this - a special type of DevOps engineer who specializes in ML solutions. As you might guess, MLOps engineers appeared due to the strong specificity of the field.
In general, an ML engineer can and should take on MLOps responsibilities if there is no such specialist on the team. I repeat, ideally, if these two specialists work together. That way, the efficiency is much higher.
Why is an ML engineer always involved in the process?
ML engineer is like a Fullstack developer for web applications. This is a person who should ideally have competencies at each stage of development in order to create an ML solution from start to finish.
In general, the competencies and responsibilities of an ML engineer depend on a company and a task. It is clear that creating a simple chatbot with the ChatGPT API does not require a large team. It is also obvious that if this is an MVP and the budget is limited, we will not be able to afford this entire squad.
As a result, there is an ML engineer in the industry - a specialist who is able to resolve all issues related to the creation and implementation of ML solutions.
As a result, there is an ML engineer in the industry - a specialist who is able to resolve all issues related to the creation and implementation of ML solutions.
In an ideal team, an ML engineer is essentially a developer of ML solutions, and is supported by a data analyst, a data engineer, an MLOps engineer, and sometimes a Data Scientist. The last one, by the way, is hired in particularly difficult cases when the expertise of an ML engineer may not be enough.
Conclusion
In the end, I would like to emphasize that in this text I answered two specific questions, which I outlined at the beginning of the article. And now, if you are asked “What does the development process for ML solutions look like?” or “What kind of team do you need and what areas of responsibility should there be?”, you can easily answer:
- “Generate a business request” (ML engineer and/or DS specialist);
- “Define a target variable” (ML engineer and/or data analyst);
- “Transform data and create ETL processes” (ML engineer and/or data engineer);
- “Create an ML model” (ML engineer and/or DS specialist);
- “Automate the process of retraining an ML model” (ML engineer and/or DS specialist);
- “Bring a model into production” (ML engineer and/or MLOps engineer).
I hope that Skynet will not enslave us, and I have explained the topic clearly and in sufficient detail. If you have any questions or comments, be sure to share them.
