Amazon ML Challenge 2024 Solution

Building an Image-Based Entity Extraction Model for E-commerce

Problem Overview

In the realm of e-commerce, obtaining precise product details from images is crucial, especially when textual descriptions are absent or incomplete. This capability extends beyond e-commerce, impacting healthcare and content moderation, where accurate data, such as product dimensions, weight, volume, and other entity values, are vital for operations.

The challenge of this hackathon is to develop a machine learning model that can accurately extract and predict entity values, such as weight, volume, and dimensions, directly from product images. This task is integral to enhancing the quality of digital marketplaces and improving user experience. The model will predict these values in a predefined format and will be evaluated based on how accurately it can reproduce the ground truth using the F1 score.

Data Structure

The dataset consists of several columns, including:

• index: A unique identifier for each product.
• image_link: A URL to download the product image.
• group_id: A category code for the product.
• entity_name: The entity value label (e.g., “item_weight”).
• entity_value: The actual value of the product entity (e.g., “34 gram”).