Comparison trial

Choosing rule-based vision or AI inspection

Compare rule-based vision and a learned model with shared defect criteria, independent test parts and a common result sheet. This example shows how to separate development from acceptance and include maintenance in the decision.

Compare two methods on the same inspection decision

Suppose a coated component must be rejected for a visible scratch while normal texture is acceptable. A rule-based tool might measure the length and contrast of narrow edges. A learned model might classify image regions from examples. This is an illustrative trial, not a report of measured performance.

The comparison should answer which method meets the agreed defect limits under production variation and can be maintained by the available team. The rule-based versus AI guide explains the broader selection criteria. Here the focus is the test record needed to make that decision.

Define labels before tuning

Write the acceptable and reject criteria with the production owner. Identify the scratch types that matter and the normal texture variation that must pass. Set aside disputed examples for review rather than assigning convenient labels. If the relevant distinction is not visible in the images, investigate the lighting or view before comparing algorithms.

Give each part a stable identifier. Record batch, material, surface finish, camera settings and capture conditions. Use the same image set for both methods where their input requirements permit. Document any different preprocessing so its effect is visible.

Separate development from the final test

Keep all images of one physical part in the same split. Where practical, separate production batches or acquisition sessions. Otherwise near-identical images may appear in both development and testing and overstate generalisation.

Use development data to configure rules or train the model. Use a separate validation set to select settings and decision thresholds. Freeze both configurations before evaluating a final held-out test set. If you change either method after inspecting test failures, report that the set has become development evidence and obtain a fresh independent test.

The exact data requirement depends on the learning method. Supervised classification needs reviewed class labels; anomaly detection may be trained mainly on accepted examples. Both still need representative defects and independent validation to assess missed defects and false rejects.

Use one result sheet

Record Rule-based method Learned method
Configuration Tool sequence, thresholds and software version. Model, preprocessing, threshold and software version.
Missed defects Defective parts accepted, divided by defective parts tested, per defect class. The same definition and test parts.
False rejects Acceptable parts rejected, divided by acceptable parts tested. The same definition and test parts.
Invalid results Missing images, processing failures and unsupported inputs. The same categories, including failed inference.
Timing Complete acquisition-to-result time under required production load. The same load, hardware assumptions and deadline.
Maintenance work Time to investigate failures and validate a changed rule. Time to review data, retrain and validate a replacement model.

Report the actual counts as well as percentages. Repeated passes of one part should be distinguishable from distinct parts. A model confidence score is not the measured production probability of correctness.

Challenge the likely weaknesses

For the scratch example, include acceptable texture with strong edges and scratches with weak contrast. Test permitted position, finish and lighting variation. Report results by condition so a good overall average cannot conceal a failure on one product variant.

Measure the whole deployed sequence with image logging and communications active if production requires them. Test missing data, timeouts and restart. Define the machine action for an invalid result independently of the chosen algorithm.

Compare the work after installation

Estimate setup, data collection, review, annotation, training, validation and integration as hours multiplied by applicable rates. Add licences and processing hardware. For recurring costs, include configuration maintenance, storage and change validation.

Test how the team restores the approved version and rolls back an unsuccessful update. Define who reviews new failure images and which changes in material, optics or production require retesting. Estimate savings only against measured current inspection, reject and intervention costs.

Choose the method that meets the agreed limits with an acceptable support burden. A combination may use rules to locate the region and a model to inspect appearance; test its combined result and total time. Bring the labelled sample inventory, failure images and completed comparison sheet to a feasibility discussion.

Vincent van Montfoort

Talk through your application.

If product appearance varies widely, we can assess whether rule-based vision, AI or a combination is the most practical approach.