Automated inspection of damaged vehicles in the auto insurance domain

Objective

The objective of this experiment is to develop a highly accurate and efficient automated system that inspects and assesses vehicle damage by leveraging advanced technologies such as artificial intelligence (AI), computer vision and Generative AI.

This experiment aims to:

1. Identify damaged part: Analyse vehicle images to identify and detect the specific parts that have been damaged.

2. Detect damage type: Classify the damage type, including dents, scratches, cracks, and structural impairments.

3. Assess damage severity: Evaluate the extent of damage to prioritise repair actions.

4. Generate damage inspection report: Generate a detailed damage inspection report with Generative AI (GenAI) that includes findings on damaged parts, damage types and severity levels.

Business use cases and applications

The automated vehicle damage inspection system has several business use cases. Some of the key use cases include:

1. Enhanced insurance claim processing: The automated vehicle damage inspection system significantly enhances the efficiency of processing insurance claims. It minimises the need for in-person inspections and streamlines the entire claims process, leading to faster payouts and improved customer satisfaction.

2. Repair cost estimation: By accurately classifying and quantifying damage, the automated vehicle damage inspection system provides information that helps in calculating repair costs more effectively.

3. Vehicle valuation and resale: The automated damage inspection system provides automotive dealers with precise assessments of a vehicle's condition, which is crucial for setting accurate resale prices. This ensures fair pricing for both sellers and buyers, which can accelerate the sales process and enhance dealer profitability.

4. Insurance fraud prevention: The automated damage inspection system helps combat insurance fraud by providing objective consistent assessments. It minimises opportunities for fraudulent claims by ensuring that damage reports are accurate and verifiable.

These use cases outline the various ways automated damage inspection systems can be utilised in business. Their efficiency and adaptability make them essential assets for insurance teams in the auto insurance industry.

Environment setup

• Python: For data preparation, AI models training and inference

• Computing infrastructure: AWS EC2 instance (GPU based instance type: g4dn.xlarge)

• AWS Bedrock: For generating reports using Generative AI (LLM: Claude V2)

Experiment approach

The different components of the automated damage inspection system that were developed are shown below:

In this experiment, we followed a systematic approach to generate an automated report on vehicle damage by implementing the following steps:

1. Vehicle image preprocessing and resizing: Enhanced the images by adjusting their brightness and contrast, and resized them for precise and consistent evaluation.

2. Vehicle parts segmentation: Trained an image segmentation model to identify and segment vehicle parts, such as the bonnet and front bumper. The trained model is then used to crop these parts in vehicles images for further inspection.

3. Damage type detection on segmented parts: Developed a damage type segmentation model to identify and classify damage types such as scratches and dents on cropped vehicle images, utilised model stacking to enhance accuracy and achieve optimal results.

4. Damage severity classification: Implemented a damage severity classification model to assess the severity of detected damage types, categorising them into minor, moderate and severe damage.

5. Data aggregation in JSON format: Created a JSON file that includes all relevant damage information, including vehicle parts, damage types and severity levels.

6. Damage inspection report generation using LLMs: Leveraged an LLM to produce a comprehensive damage report that highlights key findings and provides repair recommendations based on detected vehicle damages

Choice of algorithms

1. YOLO deep learning framework: YOLO (You Only Look Once) is a fast object detection system that identifies and classifies objects in images in one step, providing quick and accurate results. We utilised the YOLO framework for multiple tasks, including instance segmentation to identify vehicle parts like the bonnet and front bumper, object detection to classify damage types such as scratches and dents, and damage severity assessment into categories like minor, moderate, and severe.

2. Generative AI: Utilised the Claude V2 large language model (LLM) to prepare detailed damage inspection reports, summarising the observed damages and providing recommendations for necessary repairs.

Experiment outcomes

Through our experiment, we were able to automate the generation of damage inspection report for vehicles, covering details on damaged parts, damage types, severity levels and recommended repairs.

Illustrative output

The below output illustrates how computer vision identifies and classifies vehicle damages, while Generative AI produces an automated inspection report with details on severity and repair recommendations.