PSDoodle: Partial Screen Doodle

People
Christoph Csallner
Soumik Mohian
Code
PSDoodle search via partial screen doodle
WnD / TpD search via text + doodle
Data
DoodleUINet
Drawing interface used to collect DoodleUINet

Papers

Searching mobile app screens via text + doodle. 2023. (GitHub)
TpD is the first interactive and iterative search system for mobile screens that incorporates both text and sketch input. TpD's source code, processing scripts, training data, and experimental results are freely available.
PSDoodle: Fast app screen search via partial screen doodle. Proc. 9th IEEE/ACM International Conference on Mobile Software Engineering and Systems (MOBILESoft), 2022. (GitHub, YouTube: 90 seconds, 15 minutes)
Existing screen search tools are either keyword-based (Google Image Search) or require the user to produce a sketch of a complete screen. PSDoodle is the first tool to allow interactive screen search via interactive freehand sketching.
PSDoodle: Searching for app screens via interactive sketching. Proc. 9th IEEE/ACM International Conference on Mobile Software Engineering and Systems (MOBILESoft), Tool Demos and Mobile Apps Track, 2022. (GitHub, YouTube: 90 seconds, 10 minutes)
This paper adds details on PSDoodle's architecture, deployment, and evaluation.