Select Page

AI with Sensoft

 

Applies to

Sensoft Vision 2 and later

 

Question

How do I use AI with Sensoft Vision? Can I use it to automatically discard fake faults?

 

Answer

Figure 1: Activate AI on page Settings.

You can use Artificial Intelligence (AI) to categorize defects and, if you desire, automatically discard those that it infers as fake faults.

AI features in Sensoft are disabled by default, therefore you have first to enable them. To do so, go to page Settings, click on Show advanced settings, and put a checkmark next to Active in the AI section, see Figure 1.

 

Manual AI analysis

Manual AI analysis means that the operator will decide whether defects considered “fake” by AI should actually be excluded or not. On page Results, with the Select faults field (Figure 2), you can select the faults in a certain category, as inferred by AI from the photo:

  • Real, Fake, and Unsure classify according to severity
  • Lump, Dust, Hair, Neck-down and so on classify according the the fault morphology.

Figure 2: Page Results with drop-down list. The selected faults are marked in blue in the upper graph and the count in visible in the label.

In the photos, the AI model highlights each recognized feature with a colored bounding box. Features classified as real are indicated with a red bounding box, fake features in green, and unsure ones in grey. A solid bounding box indicates that the model's confidence is over 50%, a dashed bounding box that it is under 50%. When useful, a label top left of the bounding box shows the category. Since red boxes are nearly always Lump and grey boxes always Uncertain, these labels are omitted. For the precise definitions of the categories, and they they can be combined, please refer to Select faults.

Inferring many faults by AI can take relatively long, typically a second per fault. To perform the inference automatically as soon as new faults arrive, you can add the line  infer ai fake faults  in the Criteria field, which will be pretty-printed as  Infer, but do not exclude, AI-detected fake faults .

 

Automatic AI analysis (AI discards fake faults)

The AI model can automatically discard faults that it detects as fake, which is useful to prevent unwanted alarms or warnings caused for example by dust. To perform automatic AI analysis, write  Exclude AI-detected fake faults  in the Criteria field.

In this case, faults considered fake by the AI model are excluded. Excluded faults are:

  • Not counted in the Pass/Fail evaluation of the spool
  • Shown in graphs with empty grey instead of solid red
  • Kept in the record and accessible, i.e. their data and photo are not deleted.

 

Technical information about Sensoft AI

Parameters

Additional AI model settings and information can be accessed in the configuration file, which is stored in the directory Documents\LabVIEW Data\Sensoft\Config Data\Configurations Vision. The AI model parameters are:

  • AI.Active  Boolean flag that enables the AI features in Sensoft. It can be enabled and disabled in the Sensoft GUI, as shown in Figure 1. Default: True
  • AI.URL  Address with port of the web server, this port should be open in the firewall: Default: http://127.0.0.1:8000
  • AI.Model Name of the AI model. Default: ""
  • AI.Version  Version of the AI model. Default: ""
  • AI.Token  Security string required to access the AI model.
  • AI.Show on photo Boolean flag to enable bounding box around faults. It can be enabled or disabled in Sensoft by right-clicking on the photo in page Results and selecting Show AI results. Default: True
  • AI.Show percent Boolean flag to allow the software to display the confidence of the inference next to the bounding box. Default: False

Figure 3: Dialog while inferring all faults of a spool the first time.

Cache

Since inferring faults by AI takes relatively long, the results are cached, so that successive inferences a nearly instantaneous. This is why you may see the dialog window shown in Figure 3 when you infer a whole spool for the first time, but afterwards not. The cache is located at <Data folder>\Report\Cached AI results.tdms and is limited to the last 100 000 inferred faults. Each row of sheet AI results corresponds to one fault, keyed by the fault Time. Column AI result contains all bounding boxes and scores for each AI class, in JSON format. The AI classes can be retrieved from the local AI web server, and are currently [Damage, Dust, Hair, Irregular enamel, Lump, Mosquito, Neck-down, Scratch, Uncertain].

Web server

Figure 4: UI of the web server that handles AI requests.

When AI in Sensoft Vision is active, the app C:\Program Files\Sensoptic\AI\Sensoptic AI.exe is started, which creates a web server at http://127.0.0.1:8000 . You can access its API from that link, e.g. to infer single pictures. To use the API you must first authenticate yourself with your token string (you find it on the line AI.Token of your configuration file of Sensoft). In the user interface (UI) of the web server, see Figure 4, click on the Authorize button, paste the token there, click Authorize and then close the popup. You can then use the API, either directly from the UI in your browser, or by curl requests from a command prompt. The UI states the exact curl string for every request, in Linux format. On Windows replace the single quotes with double quotes and remove the line breaks (or replace the line break notifier / with ^). E.g. on Windows you get the classes of the default model with:

curl -X "GET" "http://127.0.0.1:8000/labels" -H "accept: application/json" -H "Token: c90a8ef4-4424-4607-ac80-aab78bb66500"