LADV: Deep Learning Assisted Authoring of Dashboard Visualizations From Images and SketchesMar 30 2020
This Research has been published by IEEE Transactions on Visualization & Computer Graphics, and presented by D2 front end conference.
Dashboard visualizations are widely used in data-intensive applications such as business intelligence, operation monitoring, and urban planning. However, existing visualization authoring tools are inefficient in the rapid prototyping of dashboards because visualization expertise and user intention need to be integrated. We propose a novel approach to rapid conceptualization that can construct dashboard templates from exemplars to mitigate the burden of designing, implementing, and evaluating dashboard visualizations. The kernel of our approach is a novel deep learning-based model that can identify and locate charts of various categories and extract colors from an input image or sketch. We design and implement a web-based authoring tool for learning, composing, and customizing dashboard visualizations in a cloud computing environment. Examples, user studies, and user feedback from real scenarios in Alibaba Cloud verify the usability and efficiency of the proposed approach.
By uploading sketches or images to the LADV system, users can effortlessly obtain well-designed and coded visualizations, making data mining and analysis more accessible and efficient for these groups.Examples of dashboard learned and generated from sketches We have tested sketch inputs through various mediums like whiteboards and digital screens, and in all cases, the system performed well, generating successful outputs.To accomplish this, we utilize an object recognition model to identify charts within visualization sketches, extracting information like chart type, position, and size from the uploads. Consequently, we've trained an object detection model using approximately 9,000 sketch and dashboard images.Pipeline of our approach. (a) Chart candidates are recognized from the dashboard image and then be validated through a deep learning based model. (b) Colors are extracted from the charts based on their statistical information. (c) After optimizing the layout by grid- based scheme, the dashboard is generated and interactively customized (d).Users can sketch dashboard designs on any medium, such as a sketchbook or even a napkin, and then upload these sketches to the LADV system. LADV promptly processes these sketches, generating the final coded visualization in just a few seconds.Extraction of colors from selected images For user convenience, we have developed a tool that assists users in extracting colors from any uploaded image. After color extraction, we apply the main theme color to the generated dashboard visualization, allowing users to customize their visualization based on selected images.Examples of dashboard learned and generated from images