For datasets containing a large number of short, low FPS sequences, randomly picking a certain percentage of the sequences at the original FPS will ensure an unbiased representation.
For datasets containing long, high FPS sequences, scaling down the FPS by a certain factor via systematic sampling helps extract better samples that represent the whole dataset.
Quantified metrics that help you measure quality.
Precision & Recall
Progressive algorithms measure the precision and recall of annotated datasets to help ML teams objectively evaluate data performance.
True Positive Represents correct annotations that have met all given quality parameters
False Positive Represents annotations that do not meet one or more quality parameters.
False Negative Represents objects that are not annotated but visible to the naked eye.
Detailed metrics reveal how many times an annotator is confused between certain classes and attributes while making annotations. These error metrics can help you develop better annotation guidelines and provide adequate feedback to annotators for improving label accuracies.
Different annotations require varied quality metrics.
Quality maintenance tools for efficient feedback loops
We’ve developed efficient QC tools and interactive feedback mechanisms to ensure quality standards are adequately reinforced to every annotator performing labeling tasks.
Each annotation can visually be inspected by multiple editors and built-in tools like comments, doodles, and instance-marking allow them to immediately flag incorrect annotations during the review process.
Each annotation can also be assigned for inspection multiple times and to multiple editors for air-tight quality maintenance. Our tools are designed to help the editors immediately correct mistakes or provide feedback to annotators to improve label accuracies.
Once an editor has corrected the annotation errors, a detailed auto-generated feedback report including key error metrics is sent to the annotator so that the gaps can be filled to ensure future annotations are correct and of higher quality.
High-quality data is the foundation of successful AI systems.
Built-in Quality Control Tools
With specialised QC tools like doodles, comments, and instance-marking, multiple users can run tests on samples to identify if all the predefined annotations requirements have been met with.
Custom quality check workflows
Customize your quality check workflows the way you need to achieve your desired label accuracy levels or use our built-in QC workflow model.
Project progress and performance tracking
Closely track team progress, project timelines, annotator productivity, and other key metrics via real-time analytics.
Auto-generated feedback reports
You can also provide constructive feedback to your team based on performance analytics or use our auto-generated reports to improve annotation quality.
We provide the highest quality assurance benchmarks for fully managed services.
Our comprehensive QA process includes:
High-threshold training and qualification for annotators.
ML-assistance for error reduction.
Quantitative and qualitative checks.
Customer QC and rework arrangements.
Want more information about the QA framework for our service model?