Next, we’ll apply what we’ve learned in the course by designing an AI solution related to occupational safety.
Pilot project – helmet identification
This exercise is based on an AI pilot implemented by the construction company YIT, the purpose of which was to use machine vision to identify the use of a helmet on construction sites.
The stages and content of this task don't fully correspond with real events but they have been adapted in cooperation with YIT to better suit the course.
Our starting point is that business manager Kira is excited about using AI and has identified a potential use case related to the use of helmets on a construction site. Kira asks you to plan a solution as she has heard that you have received relevant training and are familiar with the principles of AI.
Let's start the project!
Startup meeting
Kira has invited you to a design project startup meeting. She has prepared materials that explain the background of the planned solution and justify the need for it.
Here's how Kira presents her ideas to the participants:
Pilot project: helmet identification – business needs analysis
The use of a helmet is an essential safety factor on construction sites and mandatory under occupational safety regulations. If deficiencies in the use of helmets are detected and not rectified, or an accident at work occurs to an an employee not wearing a helmet, this is an occupational safety offense. In this case in Finland, the responsibility and possible penalties fall upon the individual – not the company. Any failure of an employee to wear a helmet must therefore be addressed without delay.
The use of a helmet is controlled by management, and it’s clear to all employees that a helmet is mandatory on site. However, supervision isn't comprehensive as construction sites can be large and have staff from several companies. This makes monitoring demanding.
The AI solution to be designed would support the control of helmet use by using footage from construction site cameras.
The preliminary plan for the technical solution is a cloud service where camera data is automatically stored. In the service, the AI solution uses machine learning models to identify the use of helmets.
The occupational safety team receives reports from the service that immediately show when a helmet isn't being used. The team then communicates possible shortcomings to construction site management.
Using the same image material, several machine vision solutions would presumably be feasible. In addition to helmets, the system could monitor the use of other safety equipment, for example.
The importance of the solution is increased by the fact that improving occupational safety is one of the company's strategic goals.
After her performance, Kira ponders the next step. She thinks now would be the time to apply for funding and launch the project.
Sign up to solve exercises
It looks like Kira needs to do some more research before she can apply for funding.
Creating an AI canvas
You suggest to Kira to examine the matter with the help of an AI canvas so that the use case can be viewed from a broad enough perspective. Kira gets excited about the idea, and you start compiling an AI canvas.
A) Business need and solution description
First, you want to make sure there is a business need and an owner for the solution. Kira will act as the product owner of the project.
The solution description is presented in the project presentation as follows:
The preliminary plan for the technical solution is a cloud service where camera data is automatically stored. In the service, the AI solution uses machine learning models to identify anomalies in the use of helmets.
The occupational safety team receives reports from the service that immediately show when a helmet isn't being used. The team then communicates possible shortcomings to construction site management.
Together, you decide to deepen your reflection on how to deal with the findings and come up with three options:
1) The occupational safety team receives a report on helmet use on a weekly basis and is in contact with management if necessary.
2) Deficiencies in helmet use are sent to supervisors’ phones in real time via a text message or app notification. Foremen determine whether the alarm was appropriate or inappropriate. This information is stored in a data solution that can be used by machine learning models in the future.
3) The solution uses images from cameras installed in the gates of the site. When the app detects a person not wearing a helmet arriving, a “no helmet” warning light illuminates at the gate, which also serves as a prompt for the individual to put their helmet on.
B) Availability, quantity, and quality of data
You have done some research on the availability and quality of images. Your findings are as follows:
Construction sites have signs and markings that tell visitors at the construction site that there is camera surveillance on site that is used to monitor site safety.
The cameras take video recordings that can produce enough still images to be used in machine learning training.
Different construction sites may have different cameras from different suppliers. However, all cameras produce at least high definition image quality that is sufficient for this need.
Camera data is stored locally and doesn't move onwards to a corporate cloud solution.
The cameras’ locations and distances from subjects can vary from one construction to another. However, each construction site has at least one camera at the gate.
Lighting (for example the amount of light and reflections) affects the quality of images and their usability.
After several meetings and investigations, it has been confirmed that it’s possible to send images from the cameras to the data cloud with a small delay. In addition, you have received a selection of test images from different cameras.
So far you have received answers to the following data-related questions:
What data is needed to implement the planned AI solution?
Is the data available and are the associated data pipelines in place, or should the above processes be created?
Does the required data already exist or does it need to start being collected?
Are there any known deficiencies in data quality or known data biases that could affect the reliability of the planned solution’s implementation?
As a reminder, the implementation options are as follows:
1) The occupational safety team receives a report on helmet use on a weekly basis and is in contact with management if necessary.
2) Deficiencies in helmet use are sent to supervisors’ phones in real time via a text message or app notification.
3) The solution uses images from cameras installed in the gates of the site. When the app detects an unhelmeted person arriving, a “no helmet” warning light illuminates at the gate, which also serves as a prompt for the individual to put their helmet on.
Because data issues need to be communicated to different stakeholders, you are still creating a description of the data flows in the first phase of the solution.
The figure shows how camera images taken on site pass through a planned artificial intelligence solution, where they are analyzed. The end result is reports sent to the occupational safety team regarding the use of helmets.
Sign up to solve exercises
C) Identifying helmet use through machine learning
You have submitted the test images obtained to a data scientist, who has analyzed and evaluated them before the joint meeting. At the meeting, you will explain the goals of the project in more detail and the data scientist will share their observations about the test images and the use of machine learning models.
After the meeting, Kira notes that a few things remained open.
After this stage, you are feeling good as no crucial obstacles have emerged and even the data scientists consider the planned solution to be a good target for exploiting machine vision.
D) Measuring success
Next, you’ll move onto thinking about the metrics for evaluating the success of the solution once it’s in use.
As a first measure, you evaluate the accuracy of the machine learning model. In other words, how reliably it identifies unhelmeted individuals.
Sign up to solve exercises
Aha!
The accuracy of machine learning models is just one of several possible measures. For example, it does not provide statistics on the number of helmets on construction sites, whether action is being taken on the basis of the findings of the solution, or whether users feel that the solution is useful.
The above classification accuracy alone is not yet sufficient to describe the functionality of the machine learning model. In addition, it is also important to look at what percentage of helmets are incorrectly identified as non-helmets – that is, false alarms.
Sign up to solve exercises
E) Technical solution
The next step is to consider a technical solution that includes various software solutions, cameras on construction sites, and data transfers.
At the meeting, you discuss the components and conditions needed to implement the solution with your own IT department and your camera suppliers.
The meeting reveals the following considerations, among others:
Images taken by cameras from different manufacturers are stored locally, from where they must be separately transferred to the cloud service. Data protection requirements (GDPR) must be taken into account in this process.
The cloud service being used has all the necessary components for data transfer, processing, implementation of machine learning, and reporting of results. In other words, no new software solutions are needed.
Video footage takes up a lot of storage space. You may need to consider a time limit for how long your recordings will be stored.
The IT department has found that there are no ready-made AI solutions available for this need, so the solution needs to be built from start to finish.
After the meeting, you will discuss with Kira the scalability of the solution – how it will be effectively deployed on multiple construction sites.
There are no significant problems with the technical solution. The most challenging part is transferring the footage from the work sites to the data cloud as this requires multiple technologies and parties. In addition, the capabilities of camera suppliers to help with this vary.
You will continue to work with the various stakeholders until you have a clear plan. Based on that plan, participants will know what is expected of them if the planned solution progresses to implementation.
F) Stakeholders, roles, competencies, and processes
Your research has progressed nicely and at the same time you’ve realized what kind of stakeholders and roles are involved in the project. You have made a list, including:
The company’s IT department (software, data, machine learning, etc.)
The IT provider (supports the IT department, especially with data transfers from cameras)
The company’s procurement department (camera partners, standard contracts and responsibilities)
Camera suppliers (camera selection, communication protocols)
Construction site electrical contractor (physical installation of cameras)
Management on construction sites (operational control of helmet use and monitoring of AI alerts)
Workers and visitors on construction sites (wearing a helmet)
The company’s occupational safety team (monitoring helmet use for AI alerts and informing management)
The company’s GDPR experts (GDPR compliance verification)
You guide Kira through the list, making sure you have talked to all the key stakeholders and that they have the ability and desire to be involved in the project.
After this, Kira has a question regarding processes.
G) Investment and benefits
The studies have been encouraging so far, but they have not had an economic dimension – that is, what does the whole solution cost and is it still useful enough? The investment decision will be made by management, so you decide to investigate the matter thoroughly.
Kira has clarified the various cost elements and amounts with the different stakeholders. She has compiled the table below, which shows the share of each component in the total cost of the solution.
As shown in the figure, building data pipelines and preparing the data for use cover around half the cost of the pilot project. Kira feels that this is too large, so she asks for your opinion.
Sign up to solve exercises
According to Kira, the preliminary cost estimate for the pilot project is approximately 100,000 euros, which includes the following elements:
The cost of software used (data, machine learning, reporting of results) and data cloud (storage space)
The cost of cameras for each construction site, including installation
The cost of purchasing services from external vendors, including building data pipelines from the cameras to the data cloud, data processing, and implementing the machine learning models
The cost of internal work isn't included in the calculation. Internal work includes:
Project management
Work with external suppliers on data processing, machine learning, and reporting results
Training the occupational safety team and management selected for the pilot
The estimated amount of internal work input is 120 person-days.
Following the pilot project, the implementation of the solution on the new site will cost an estimated 25,000 euros and will require approximately 30 person-days of internal work.
However, Kira wants to discuss with a new AI vendor who is proficient in Generative AI to understand whether the latest GenAI technologies e.g. GPT-4 can be used to understand images and detect if a person is wearing a helmet or not. Together with the vendor, she does a quick prototype by uploading images to the OpenAI GPT-4 service and building a prompt that aids in image understanding and helmet detection. The output with different kinds of images gives her some evidence, without spending a huge sum of money, if this could be the right approach. She wonders if the project implementation cost can be significantly reduced by leveraging this technology. However, she also wants to understand the operational costs per usage of this technology, so that she can make a comparative analysis between these approaches.
“Good job,” adds Kira, who notes that this was the easier side of the investment calculation. Next, the benefits of the investment should be justified.
Justification of the investment
As usual, there is no pre-allocated funding for such a pilot project. So it will be up to Kira and you to secure it.
You will soon find that the actual investment calculation fails to justify investment because the solution doesn't generate direct revenue or savings that could be reflected in the cost of the solution. The benefits and rationale for the solution must therefore be found elsewhere.
You promise Kira that you will spend the next day thinking about a good investment rationale that makes it easy to convince management of the project’s importance.
Decision
A week later, Kira takes you to a meeting where she will present the solution to management and apply for funding and a startup permit.
The solution is divided into two stages, the first of which is implemented according to option 1 described in the construction of the AI canvas. At this stage, the foundations and technical functionality of the solution are built and the various parties have been brought into the project.
After the first stage, an extension is made to continue to option 2, which involves management and makes the solution real time. This creates significant added value for the solution.
During the meeting management asks questions, some of which Kira addresses to you.
At the end of the meeting, you have received funding and permission to start the project. There are many dimensions to planning an AI project, but now you have created the right conditions for a successful implementation. Good luck!
In the third chapter, you learned about the following:
One of the biggest obstacles to the use of AI is finding suitable use cases.
How to identify use cases for AI in business environments and how to assess their feasibility using evaluation matrices and an AI canvas.
The ten areas of a typical AI canvas. Of these, business need and data are particularly important.
The benefits and costs of a solution need to be looked at extensively. The benefits can also be qualitative, for example related to occupational safety.
The typical roles in an AI project are product owner, data engineer, data scientist, and project manager.
To apply what was learned during the course, together with Kira, you designed an AI project to use machine vision to control the use of helmets on a construction site.
