Getting your data culture right
How your analysts and developers work together is as vital as the data you get. A corporation with departments living in silos will not see the full benefit of your investment. If a focus on efficiency is essential to your business, a focus on how your data team interacts with each other is vital.
The Business / IT Relationship
Business wants answers tomorrow; IT needs a few weeks to develop. Every business has the same story. The business side focuses on getting answers immediately to pivot and take action. The IT side wants time to investigate, develop, test, and deploy. The IT process is time-consuming, but it is needed. Every IT team wants the time because they have been burnt before by deploying something before it was ready and getting hammered because the data wasn't correct. The relationship can strain because of this, but both sides must understand each other's motivations. One side wants to know who to yell at; the other doesn't want to be the people getting yelled at! In the most cohesive environments I have been in, the Business Intelligence (BI) team has one person who sits on the IT team's daily stand. They will report to the director of BI and ensure that all work is prioritized appropriately and that projects will meet these deadlines. It is essential to ensure that someone is sitting in on these meetings because there are typically questions on how the data should be, and emails can get skipped over. In a perfect world, I recommend that the warehouse data engineers report to the BI lead or both teams report to a Chief Data Officer. This way, the business and IT are always aligned.
Creating a Champion Program
The champion program is a broad group of data analysts who work for the company. While it sounds cheesy, these programs work and make everyone feel actively engaged. Feeling you are part of a broader team gives each analyst a more profound sense of belonging and saves them time because they are more likely to know who to go to when they have questions. I have found it helpful to have monthly meetings where the Champions in the company, along with data leaders, such as directors and VPs of data dependant departments, can join and hear the current data issues, what was released last month, and what will be released in the coming month. Analysts need to be informed of changes because downstream impacts can happen with every change you make. The broader the group and the more ears hearing your message, the less likely a change will impact the data environment.
The Data Masters
One additional program I have created recently is the data master program. I have always tried to have meetings with directors and up on a bi-weekly cadence. Most of them are excited to join the company's data culture. However, after a few meetings, they lose interest in hearing the BI team talk about tables, views, and reports. I have found it more beneficial to go down and engage with the power analysts in each department. I now meet with this group of people bi-weekly and inform them of data work in the pipeline. The data masters will know the impacts of what you are doing, and they are trusted within their department. Their primary role is to share the information up the chain. It's a big ego boost to approach analysts and commend them on their knowledge of the company and their work output. Analysts are usually excited to participate and engage however you need them to.
Data Badges
A fun program we have done is departmental data badges. Each Data Master helps create a basic test that covers all the nuances and rules of their departmental data. They know what an average analyst within their department should know. Analysts have an opportunity monthly to take the test and prove that they know their stuff and are a reliable resource when people need analysis. These tests are a sneaky way of collecting names of people in the company who have the analytical chops within each department to help out when needed. Stage two of the data badges is finding and creating a group of Subject Matter Experts (SME) within each department. After an analyst has had the basic badge for a set amount of time, they can complete a project, proving they know the ins and outs of that specific department. The project should consist of the analyst being able to work with the raw data, clean it up, join it to corporate-level data, and create a report or presentation on their findings. The project should focus on very granular parts of the department data. After the analyst has completed this, you know you can rely on them for heavy analysis that isn't necessarily in pre-defined tables. We have created badges that the analysts can put in their email signatures so that when they provide analysis, the receiver knows they can trust the results.
BI Daily Stands
I understand that no one likes to have meetings all the time, but I firmly believe in your primary analyst hosting "office hours" for the rest of the BI team. These office hours are not required for anyone but should be highly encouraged. So many analysts get stuck in the weeds and spend too much time trying to finish a project because they don't want to admit they don't know or don't know whom to ask. Having a casual time where people can drop in and ask questions can generate efficiency by allowing the analysts a no-judgment zone to ask their questions and potentially find the missing piece that will enable them to finish their analysis. If you don't have a primary analyst, look for someone tenured who understands the data. Many days, people join, and we spend ten minutes chatting about a new movie or discussing a YouTube video on Power BI. While this may sound like a waste of time, allowing people to acclimate in the group has long-term benefits. It may seem like a stereotype, but most analysts I have met are either introverted or overly confident in their abilities. Neither group will ask for help unless they feel they are in a judgment-free zone, and spending a little time letting them know you care about them just as much as their abilities pay dividends.
Two Week Sprints
I have worked in environments where there is a list of things to do, and you grab and go, and I have worked where an assignment has a 2-week deadline. If a project should take longer than two weeks, benchmarks are created. In my experience, the 2-week sprint system works the best. High motor analysts want to grab projects and finish them in record time to prove their worth to their boss and themselves. I have found that this group aligns more with imposter syndrome than others. They want to churn out work at a pace that keeps them away from trouble.
The major problem is that 9 out of 10 projects will be completed ideally and in a record time. However, the 10th project will make an incorrect assumption, invalidating the work results. By having two-week sprints, the analysts must slow down, validate, and reread their code to ensure no mistakes. It may seem like a lack of efficiency to slow down a great mind. However, you don't want to write a mea culpa letter to the CEO because they moved forward with an initiative based on incorrect analysis. Your slower, more methodical workers also work better in this environment. When you don't have these sprints, projects will run over, and timelines don't seem stressed as much.
During these two-week sprints, giving everyone one major and one minor project is a good policy. It would help if you encouraged your analysts to step away and work on their minor after a few days of complex analysis. Sometimes, an analyst gets so in the weeds of their project that they miss the big picture. Have your BI meet once a week, with one week giving out the projects for the next sprint and the off week giving them time to discuss with the group where the project is. Analysts need to be open to criticism and allow others to voice any concerns with their work.
Release Dates and Communication
Suppose your company creates reports that go to corporate stakeholders or your locations. In that case, I recommend setting a day for releases and communication. In a perfect world, IT releases any bug updates or new tables into production on a Wednesday morning, and the latest reports go live on Wednesday afternoon. Everything will flow seamlessly if the releases are done correctly and new tables were previously validated in a development environment. Communication on what is coming out should go out on Monday so stakeholders know about changes or additions they can prepare for. I like the Wednesdays for release because they are in the middle of the week. Everyone is focused on last week when they come in the office on Monday; your C-Level people have analysts answering this week's burning questions on Tuesday, and releasing on Thursday or Friday could lead to bugs you missed being found on the weekend. No one wants to log in and do Power BI on the weekends! Having a set day, people can expect the changes, and setting it at the right time of the week makes a world of difference for reliability, peace of mind, and work-life balance.
