BLOG Machine Learning

Automation: The Antidote to Overcoming Labeling Inefficiencies

Akshay Lal
April 8, 2020

As more and more technologies are emerging with AI foundations, the global datasphere is continuously expanding. This improved data accessibility is a definitive advantage for developing cutting-edge autonomous systems. However, the paucity of accurate, labeled data will substantially slow down AI advancement. 31% of IT decision-makers reported that slow data performance was a big obstacle to their data strategy; this shows how the inadequacy of high-quality labeled data can substantially slow down development. Most companies dealing with large datasets have problems achieving desired levels of accuracy with data labeling. And one of the most obvious reasons for this problem is increased reliance on human intelligence to execute tedious and complex annotation tasks. 

Manual Data Labeling: The Biggest Barrier to AI Advancement

Enterprises heavily rely on human intelligence to acquire training datasets for ML models. When we first built annotation tools for data labeling, they were completely manual. The annotators would draw boxes after boxes and mark points after points without any feedback or assistance from the tool. Manually drawing and annotating a single box hardly takes any time, but when millions of boxes enter the equation, the time and effort involved add up very quickly. 



The manual process is time-consuming, labor-intensive, and scaling would require huge investments, making it expensive. Further, the quality of the labels is influenced by various factors like tool capabilities, workforce effectiveness, tool UI, data complexity, number of classes of annotations required, etc. The list could go on; the variable factors affecting the process are way too many. 


Enter Automation - The Antidote to Overcoming Labeling Inefficiencies


Automation has always been the answer to by-passing the inefficiencies involved in fully-manual operations. Therefore, when it comes to data labeling, with semi-to-sometimes-fully automated tools, acquiring large swathes of diverse, high-quality ground truth datasets can be executed faster, at reduced costs, and with improved accuracies.


At Playment, we believe ML-assisted labeling tools can help overcome these barriers of scale, quality, and accuracy, facing AI’s most underrated workforce. 


Our ML-assisted labeling tools are guided by 3 important principles:



Deep Dive: How does Playment ensure faster, accurate, and efficient labeling? 

Here are a few illustrations of how Playment’s ML-assisted innovations can simplify the execution of complex and tedious annotation tasks. 


Interpolation for Video Object Detection and Tagging


We use interpolation methods to label objects in a sequence. With the interpolation feature, the annotator is required to label every second to the fifth frame in a sequence, instead of labeling the same object across each frame. This drastically reduces the time taken to label videos and sensor fusion sequences. 


Before: Annotators Manually Draw Cuboids For Each Frame (Number of Cars Annotated = 3)


After: Interpolation Automatically Detects Objects In Multiple Frames Increasing The Number of Cars Annotated From 3 to 6



Interactive Instance Segmentation

Semantic segmentation can be executed in mere clicks. By marking the extreme points of an object, the tool automatically generates a semantic mask. This speeds up the segmentation of objects by 10x. 


Before: Annotators Manually Plot Many Points For One Polygon


After: Annotators Only Need To Plot Four Extreme Points For One Polygon



One-Click Cuboids

Cuboids in 3D point clouds can be drawn with just a click. When an annotator clicks on a cluster of points, the pre-trained model automatically identifies the best fitting cuboid. This reduces the time taken to execute 3D point cloud annotations by 25%. 


Before: Annotators Manually Draw and Drag Boxes To Make Cuboids (Number of Cars Annotated = 3)


After: Annotators Use A Single Click To Make A Cuboid Increasing The Number of Cars Annotated From 3 to 6



ML Proposals for Computer Vision Scenarios 


Our semi-automated and highly-interactive annotation tools enable faster and more accurate labeling with  lesser number of clicks and ML-assistance for checking the quality of the annotations. Our proprietary labeling models are developed based on state-of-the-art machine learning architectures and are trained on a variety of datasets. 


Generic models trained on common object datasets can be used in a variety of computer vision scenarios like object detection, object tracking, 3D-object detection, semantic segmentation, etc. Specific models trained on autonomous vehicle datasets are used for AV-related use cases. Our annotation tools allow annotators to view model proposals, play around with thresholds and select the proposals which are accurate and either reject or edit the annotations. 


So, what are the advantages of an ML-Assisted Annotation Tool? 


To summarize, we believe that an ML assisted annotation tool can help you deploy your models faster as a business. 


Shorter Annotation Timelines: With ML-assisted annotation tools, data labeling speed can be increased up to almost 40 - 60%, vastly reducing the project timelines for companies building complex ML models. 


Lower Annotation Costs: With pre-trained models, the annotators spend far less time executing annotations because the models eliminate any unnecessary labeling tasks and can focus their efforts on labels that display a low confidence score. This saves time, which in turn reduces the annotator costs involved in data labeling. 


Higher Labeling Accuracies: The time saved by the automation can now be spent by human labelers to fix any errors that might have cropped up. This helps in improving the accuracy of the labels. The human annotators further help improve our models by marking inaccurate annotations that can be used to retain the models.   


With managed time, cost, and quality, businesses can scale their AI projects from tens to thousands to millions of data-points without any added hassles.

What Does The Future Entail?


This is only the beginning. With the mission to expedite the AI age, we are constantly improving our model accuracies to efficiently funnel the ML pipelines of companies building cutting-edge technology. 


In the next few months, Playment will be releasing top-notch automated software infrastructures that will enable more efficient annotation execution and easier project management for data labeling. The platform will offer more unique features along with the ones I previously discussed. Here’s a sneak peak of what we have in store - improved ML-assistance, holistic analytics, workflow management  tools, elaborate quality check processes and tooling, and everything in between to streamline ML pipelines. If you’d like to get updates of our latest features and what we are upto, share your email with us here