Effective Data Discovery and Exploration Strategy

Data Visualization

Everything starts with having the ability to visualize the end product. The more often users can visualize the end product, the more likely the end product will match their business needs. Traditionally, requirements are gathered, written and reviewed with end users before a product is built. Mock-ups are created to help users visualize what they will get, but when they actually see the end product is when they come up with change requests to make the product closer to what they want.

In Traditional Business Intelligence tools, Data has to be organized and distributed before any visualization or discovery can be done. In emerging Business Intelligence tools like Qlik Sense, visualization is done at the beginning and at every step of the product development enabling back to back feedback between users and developers. Data visualization begins once the data has been extracted and through out the rest of the extraction, transformation and load process and data anomalies are discovered at an earlier stage in the development process. The advantage with this approach is the great amount of time and resources are saved on building something that fulfills the real requirements. The business receives a working prototype within a matter of days. The prototype is reviewed using interactive visualizations and refined through out the data discovery process.

Teams & Roles

As data visualization enables users to explore data and provide feedback to research and development team, a data governance team must be in sync to support the greater data strategy of the business.

At minimum, the Data research and development team must include analysts, designers and programmer. The analysts will use their knowledge of the business to act as product owners interfacing directly with users. The designers will work with the analysts to extract and build rich visualizations. In specific cases where custom visualizations are needed, the data programmers become involved. To make statistical sense of data or to build predictive models, data scientists are involved to extract knowledge from data.

Once a stable solution has been built, the solution is transferred to the data governance team. The data governance team will continue to collaborate with data research and development team as needed. Since data will be used end-to-end by a business, data governance ensures that business rules are enforced, accurate and proper data definition is maintained, and data is kept secure. At minimum, the Data governance team must include the governor, designer, administrator and educator. The governor is responsible for enforcing business rules, security and accessibility. The administrators will support governors, designer and users as well as perform maintenance activities. The educators will continuously train users and help them understand visualizations. The data governance team will ensure clear enterprise communication and enforcement of data visualization standards as users will begin to perform their own data analysis.

Agile Analytics Development

While all teams have their own set of roles and priorities, taking advantage of agile project management methods has proven to be effective. The agile project management focuses more on interaction between users and developers and a working software rather than the tools, processes and heavy documentation. Requirements are written from the stand point of what users want, why they want it and what goal is achieved. Developers would then write the code, show it to the user and refine it further after learning more from the data.