Neural Network Symbol Recognizer

Demo: LINK Tutorial: LINK

Plugin enables you to draw collections of Symbols/Patterns which can be recognized during gameplay by a Neural Network system.

It can be used in different gameplay scenarios:

• Cast a spell according to a mouse gesture
• Paint a symbol on doors and unlock them if correct
• Make a minigame that requires speed and precise drawing to pass

Basic functionality:

• Draw different patterns (Symbols) in a plugin's window. It can be letters, digits or any abstract shape. Launch machine learning and save the result. Test drawing accuracy inside the plugin window. Then do a simple setup in Blueprints or code to be able to draw and test the drawing accuracy in a game.

Workflow for in game drawing and testing symbols accuracy:

• Initialize drawing by calling a method BeginDrawing() on input PRESSED
• Pass brush (cursor) location into Draw() method when moving your brush or in Tick
• Call EndDrawing() to break current drawing line on input RELEASED
• When a drawing is finished Test its accuracy by calling TestDrawing or GetMostAccurateSymbol
• React somehow to the test result (either kill a player if failed of throw a blizzard of fireballs when succeeded)
• Call ClearCanvas when current drawing is not needed and want to start new drawing.

Plugin contains 3 examples of In Game usage (SRTestLevel)

Technical Details

Features:

• Built-In tool for drawing and saving symbols.
• Customizable parameters to modify Neural Network sensitivity.
• Simple setup for in game use.
• Example level with 3 mechanics (casting spells, drawing on mesh, minigame).
• Step by step tutorial inside the plugin.
• Drawing symbols is independent from size and position which makes it much easier to draw a correct symbol.
• Two debug widgets to test drawing accuracy in game.
• Possibility to create multiple symbol collections that can be set for recognition in gameplay.

Code Modules:

• SymbolRecognizerPlugin (Runtime)
• SymbolRecognizerEditor (Editor)

Number of Blueprints: 8

Number of C++ Classes: 20

Network Replicated: No

Supported Development Platforms: Win64/Win32

Supported Target Build Platforms: Win64/Win32

Documentation: 'ShowTutorial' inside plugin or partly in video tutorials

Example Project: Inside plugin SRTestLevel (SymbolRecognizer/Resources/TestContent)

Important/Additional Notes: Do not change default root plugin folder name. It must be 'SymbolRecognizerPlugin'.