Project Goal:
From Search to Scalable Data Management Platform
Project Summary
Unifying Data Management Tools into a Central Platform
For years, the data apps tools ecosystem has been disconnected and hard to use. UX breakthroughs like the creation of DataHub, a central search portal, has improved usability and speed of development, but user's tasks are still disjointed, and it takes a long time to build new tools and technology.
Problem:
Data Workers (tools users) have broken workflows across many apps, and sometimes it's unclear what to use for what tasks. Learning how to use the many apps takes a long time, and require many tedious repetitive manual tasks.
For Data Developers (people building the tools) it's hard to know how to build new tools. It often takes over nearly a year to build new tools, so teams often sacrifice Usability below functionality in shipping the MVP. However, developer teams' priorities change after shipping clunky MVPs, so the tools often do not reach maturity or elegance they were designed to achieve.
Process
Discover - Define - Design - Deliver
Solution
From Search to Platform: DataHub UI will expand beyond the role of central search to be a single pane of glass for all data management workflows, where new tooling, like managing compliance and data quality are build in seamlessly with a consistent interface using a reusable components and common design system. Existing tools, like Azkaban, Darwin Notebooks connect through intelligent links
Information Architecture:
Previously DataHub 1.0 was structured around search: the ability to find, view, and take action on data objects, like datasets, metrics, and more.
DataHub 2.0 expands beyond search to support data management tasks and workflows, and creates a scalable way to add in new technology, tooling, and workflows for years to come.
Quality & Execution
Reusable Modular Components & Page Layouts: The new system elevates the quality of all the tools by sharing an open source design system and front-end. Plus the project created a modular strategy to mix and match reusable components
Outcomes
With this system, design time can shorten from quarters to hours, development of new tools can shorten from years to quarters or sprints. The team can
Information Architecture sets the long term UX strategy
Project Details
Problem:
Hard to build, hard to use.
There are over 20+ engineering teams in the Data Apps department at Linkedin. Historically, the team building the technology is also the one building the app. This leads to a disconnected ecosystem built in silos.
Hard to build:
Many data engineers are unfamiliar with UX/UI, and have no resources how to build
Hard to use: The wide range of data apps are often built without designers or standard UI behaviors. They are often difficult to use and sacrifice usability for technical capability.
Objective:
Easy to build, easy to use.
In 2022, the DataHub UI team decided to solve how all these apps can integrate smoothly to build better user experiences and speed development.
Easy to Use:
A single pane of glass that lets users access all of their data apps, workflows, objects & products all in one place. Users can manage their job-to-be-done across multiple workflows, which flow so naturally together that they may not know they are built by different technology teams.
Easy to Build
A scalable platform that can infinitely add new features and workflows as new technology teams build new. A clear architecture helps both central and extension dev teams know how to plug-in the features into the integrated whole.
Personas
Who does this project help?
Data Developers: Information Architecture helps clarify the needs, opportunities, and future of a product. The immediate users of the design deliverables are the development teams that have a clear understanding of how to work together, where and how their features can integrate, and how to use standard components.
Product Users: Many apps appearing to users to function as one product helps end-users, who can ultimately use a clean, intuitive, and consistent UI that connects all their data management tasks in one place. With a clear working model, the dev teams can ship features and apps quicker, ultimately improving the power of the tools ecosystem.
Design Process
I self-led this project following a common design process, including:
1. Discover: Understand the Problem
Gather the wide range of needs through user research, digesting all data departments roadmaps, and industry practices and more to understand the breadth and depth of the problem.
Discovery included:
User Research - 2 long-term projects by one by a senior designer, and one by the product PM
Departments and Feature Requirements Docs
- 5 Department Roadmaps - written by Senior Directors set long-term vision and goals for their departments.
- 20+ Technical Project Requirements - written by senior engineers and data scientists about the technical tools and technology they need to build in coming and past quarters
2. Define: Set the Requirements
Identify what is really needed to solve the problems systemically.
This includes being infinitely scalable, modular, and feasible to ship with quality and speed
3. Design: Solutions
Brainstorm lots of ideas and possible solutions, gather partner and user feedback, refine and iterate, iterate! Design deliverables include:
Sitemaps - Expand the sitemap to better facilitate user tasks, alerting, monitoring and the producer workflows.
Wireframes - pages layouts with modular components
4. Deliver: Results
Refine ideas and to narrow into a feasible solution. Share, and help shift company direction for years into the future
lala
Design Deliverables
Sitemaps
Map of All Current Objects
Including their properties, metadata, and sub-pages
Map 1: Current Sitemap & User Flow
The current map focuses on finding, viewing, and acting on objects.
Map 2: Extend for Jobs to be Done
After studying the product needs across 2 user research studies, 5 Data Departments long term roadmaps, and 20 upcoming feature requirements, I created an scalable sitemap the create a way for the site to address all known roadmap needs, AND is scalable to meet unknown future objectives.
Map 3: Expand with Planned Verticals
The structure must be flexible to meet both planned and unknown future needs. So the mental model of adding work centers (apps) that can cross the knowledge graph from any dimension is infinitely scalable.
After creating sitemaps for the system, it's time to create page layouts for all future pages
How to ship quality quickly?
Quality & Execution
Ship elegant products using reusable components & workflows built on an open source design system
Problem:
Lots of technology teams need to build new features, but have limited UX or Front End experience or headcount. This results in sacrificing design quality to prioritize technically functionality.
Solution:
Off-the-shelf pre-build UI components and page layouts so that teams can leverage existing patterns and customize only as needed.
Off-the-Shelf Components & Page Templates
With mix & match components, we can build templates & layouts for all new pages identified in the future sitemap. Any page can inherit standard components, like alerts, success stats, a list of tasks, a list of objects, or read-only metadata content, or customizable analysis.
Mix & Match Content Types
- Which components are part of the roadmap? Developers can build content that is feasible or a priority.
- What layout is best for users? Can rearrange content blocks to be best for target users' needs, feedback, hierarchy, etc.
- Placeholders for future releases - Clear space for future features to be added, and not break the layouts.
Common Page Layouts
Discovery: Search, Browse, Explore
Previously, Search was the only way to find things. But it was often hard for users to find things because:
Search's Problems & Limitations:
- Users needed to know already what they were looking for: including exact names, strings, or keywords
- It was hard to know which dataset was the right one, due to duplications, locations, or similar
Solutions: Better Browse
Add more ways to browse content so users can find things even when they don't know exactly what they are looking for. This includes discovering featured content, like Data Domains, browsing through relationships, like org structures, and a more robust search.
Discover by Search
Discover by Browse Directory, Line of Business, or Featured
Discover by Team Org & Ownership
Management Portals
Management Portals structure content and functionalities around the user's workflows. Users can easily see what they need to do, and easily access the actions needed to perform their jobs-to-be-done
Mix & Match components can be rearranged to fit product requirements and & user needs.
My DataHub - Producer Zone
While many analysts, novice and intermediate users focus on discovery, producers of the data need better ways to manage the data they own. Currently, there is no easy way for them to see all the objects they own, know if any actions or problems need to be done, and easily access the workflows to manage those objects. With the new sitemap, producers can easily see 1. Discover: what they own, 2. Manage: see all the functionalities & actions on their data. Eventually, the producer pages can also inherit other standard content types, including 3. Alerts, 4. Monitoring Success Stats, and 5. Links to outside workflows and apps.
My Ownership: Manage What I Own
MyDataHub > Objects I Own > Datasets I Own > Edit a Dataset
My Tasks: Manage All My Workflows
My DataHub > Alerts > Management Overview Portal > Feature Management Portal
Project Results
Easier to Build
With this architecture and set of wireframe templates, teams can quickly identify project needs, leverage off-the-shelf designs and UI, and ultimately ship higher-quality products at speed and at scale. This system is also future-proofed to be able to expand in multiple dimensions to accommodate known and unknown future growth.
Faster Project Diagnosis
1. Where to build new features on the map?
2. How to design the apps using reusable layouts and workflows?
3. Which components to use off-the-shelf vs. customization?
4. Who should build which part of the project?
Clearer Collaboration and Resourcing
It is easier to see which dev team builds which parts of the project. It also removes redundancies and overlapping work between the dev teams.
Faster UX Design
Speed design time down to hours or weeks instead of months or quarters, speed development time,
Faster Front-End Development
By leveraging reusable UI components with open-source design systems Ant and React, and micro-front-ends, developers can cut out hundreds of hours previously consumed on custom, non-standardized components.
