When Algorithms Aren?t Enough

When Algorithms Aren?t Enough

Algorithms seem to be magical.? One only needs to upload a new photo to their Facebook page and instantly the people within your photo are captured and tagged.? We can assume that if Facebook offers this feature for free, a simple investment in a similar facial recognition algorithm should yield worthwhile results. ?But the stakes for photo tagging for social media are much lower than for identifying persons of interest in an investigation. Both scenarios include video and image content analysis, but with very different outcomes and different consequences for an incorrect match, or false positive. Yet many technologists mistakenly believe that the same algorithm can solve for both.

Human Cognition + Computer Vision and Learning

Computer vision and machine learning are wonderful tools. , But to ensure a higher degree of accuracy for high stakes scenarios they must be combined with the power of human cognition. Algorithms solve the issue of scale but by sacrificing accuracy and flexibility.

Algorithms are simply coded instructions telling a computer what to do.? Some algorithms perform functions, while others choose function-performing algorithms.? Human cognition can take cues from algorithms, and people take over where algorithms leave off.? A person recognizes a face in a video or image and tags them as such, then the facial recognition algorithm takes over. Human cognition will feed algorithms data that algorithms should then successfully process.? The trick is to combine the two in the most innovative, most flexible ways possible.? There are programmers, students, and businesses that are developing many millions of these algorithms, but the algorithmic job of the future is to attain the knowledge to determine which 14 of the 10 million algorithms available should be applied to my scenario.? Having humans in the loop during the entire workflow is the only thing that makes this possible.

As I said before, human cognition struggles to scale but new tools and technologies for parsing out the workflow and the analysis tasks, plus the ability to crowdsource the people power, goes a long way in solving the scale issue.

For example, let?s take a security risk scenario: identifying a person of interest.

Input: Unstructured data ? Surveillance video or body camera

  • Step 1: Detect faces ? Run facial recognition algorithm to pull all faces out of the video creating a set of images. Algorithms are very accurate at this.
  • Step 2: Face image dataset analysis ? goes to crowd source because we need a high level of accuracy to correctly match a face against a Be on the Lookout For (BOLO) database
  • Step 3: Final results ? matches are sent to law enforcement.

In this case the algorithm is employed in the workflow to do what it does best, creating a reliable dataset for further analysis.? People are then brought in to do what they do best. The human cognition is crowdsourced so a large resource pool can be created to turn the job around faster, and released when the job is complete.

Creating flexible workflows and tools are the key to combining algorithms and human cognition to create highly scalable, yet highly accurate smart vision workflows.