1 00:00:00,040 --> 00:00:03,720 Welcome to the M365 FM podcast, where we explore the people, 2 00:00:03,720 --> 00:00:06,639 ideas and technologies shaping the future of work. 3 00:00:06,639 --> 00:00:08,720 Opening the cost of copying data. 4 00:00:08,720 --> 00:00:09,839 Picture this one. 5 00:00:09,839 --> 00:00:10,880 Sales table lives in a 6 00:00:10,880 --> 00:00:12,560 warehouse, a lake house, a power 7 00:00:12,560 --> 00:00:14,240 BI import model, and three team 8 00:00:14,240 --> 00:00:14,919 folders. 9 00:00:14,919 --> 00:00:16,239 That first copy started as a 10 00:00:16,239 --> 00:00:17,719 convenience, but pretty soon the 11 00:00:17,719 --> 00:00:18,640 numbers disagree. 12 00:00:18,640 --> 00:00:21,920 Reports go stale and someone inherits another refreshed job. 13 00:00:21,920 --> 00:00:23,480 Storage isn't where the pain lives. 14 00:00:23,480 --> 00:00:24,399 Trust is. 15 00:00:24,399 --> 00:00:28,640 Welcome to the M365 FM podcast, where we explore the people, 16 00:00:28,640 --> 00:00:31,280 ideas and technologies shaping the future of work. 17 00:00:31,280 --> 00:00:32,399 We're looking at one lake and 18 00:00:32,399 --> 00:00:33,799 this idea of shared data, 19 00:00:33,799 --> 00:00:35,240 because one stored copy can 20 00:00:35,240 --> 00:00:36,439 still support every team's 21 00:00:36,439 --> 00:00:38,119 preferred way of working without 22 00:00:38,119 --> 00:00:39,039 the mess. 23 00:00:39,039 --> 00:00:40,880 The mess one lake was built to stop. 24 00:00:40,880 --> 00:00:42,439 Most organizations never planned 25 00:00:42,439 --> 00:00:43,640 to build that maze of data 26 00:00:43,640 --> 00:00:44,000 copies. 27 00:00:44,000 --> 00:00:45,439 It starts innocently enough. 28 00:00:45,439 --> 00:00:48,920 One team trying to solve a real problem fast finance needs a 29 00:00:48,920 --> 00:00:52,119 sales report, so they pull the data into their own workspace. 30 00:00:52,119 --> 00:00:55,679 Operations needs a daily view, so they spin up another pipeline 31 00:00:55,679 --> 00:00:58,840 product, wants to study customer behavior, and their team builds 32 00:00:58,840 --> 00:01:00,310 a lake for that work. 33 00:01:00,310 --> 00:01:01,509 Each of those decisions looks 34 00:01:01,509 --> 00:01:03,189 reasonable on its own, but when 35 00:01:03,189 --> 00:01:04,469 you step back, you get separate 36 00:01:04,469 --> 00:01:05,870 lakes, separate pipelines, 37 00:01:05,870 --> 00:01:07,150 separate permissions, and 38 00:01:07,150 --> 00:01:08,390 separate versions of the same 39 00:01:08,390 --> 00:01:09,390 business fact. 40 00:01:09,390 --> 00:01:11,870 Take a simple question how much did we sell last month? 41 00:01:11,870 --> 00:01:14,390 The SQL team might grab the number from a warehouse. 42 00:01:14,390 --> 00:01:16,230 A data engineer pulls from a spark table. 43 00:01:16,230 --> 00:01:19,549 After a transformation job, an analyst opens a power BI report 44 00:01:19,549 --> 00:01:21,109 that refreshed overnight. 45 00:01:21,109 --> 00:01:23,829 A business app reads from its own extracted data set. 46 00:01:23,829 --> 00:01:27,150 Those answers should match, but sometimes they don't. 47 00:01:27,150 --> 00:01:30,310 Picture this one pipeline runs at six in the morning, another 48 00:01:30,310 --> 00:01:32,069 fails and retries at noon. 49 00:01:32,069 --> 00:01:33,349 The warehouse excludes canceled 50 00:01:33,349 --> 00:01:34,989 orders, but the report uses an 51 00:01:34,989 --> 00:01:35,870 older rule. 52 00:01:35,870 --> 00:01:38,629 The sales team corrects a record in the source system, yet only 53 00:01:38,629 --> 00:01:41,430 one out of five downstream copies ever sees the change. 54 00:01:41,430 --> 00:01:43,750 Suddenly, the meeting isn't about sales anymore. 55 00:01:43,750 --> 00:01:46,790 It becomes a debate about which number you can trust. 56 00:01:46,790 --> 00:01:48,790 That debate costs more than storage space. 57 00:01:48,790 --> 00:01:50,189 People burn hours tracing 58 00:01:50,189 --> 00:01:51,510 columns through pipelines, 59 00:01:51,510 --> 00:01:53,030 comparing refresh timestamps, 60 00:01:53,030 --> 00:01:54,230 and hunting down the person who 61 00:01:54,230 --> 00:01:55,590 owns a table nobody remembers 62 00:01:55,590 --> 00:01:56,430 creating. 63 00:01:56,430 --> 00:01:59,040 Meanwhile, each extra copy demands access controls, 64 00:01:59,040 --> 00:02:01,530 monitoring backup plans, and someone on call. 65 00:02:01,530 --> 00:02:04,689 When the load breaks, the technical debt piles up quietly. 66 00:02:04,689 --> 00:02:06,290 A pipeline depends on a table. 67 00:02:06,290 --> 00:02:08,050 That table depends on another copy. 68 00:02:08,050 --> 00:02:09,330 Someone changes a column name 69 00:02:09,330 --> 00:02:10,810 upstream and a report breaks two 70 00:02:10,810 --> 00:02:11,650 systems away. 71 00:02:11,650 --> 00:02:13,129 The fix takes ten minutes. 72 00:02:13,129 --> 00:02:15,490 Finding the responsible person that can take days. 73 00:02:15,490 --> 00:02:18,289 You might think the answer is forcing everyone into one tool. 74 00:02:18,289 --> 00:02:21,050 Lock every team into SQL mandate, spark for every 75 00:02:21,050 --> 00:02:23,650 engineer, or enforce one model for all reports. 76 00:02:23,650 --> 00:02:26,289 But here's the head fake that doesn't solve the real problem. 77 00:02:26,289 --> 00:02:29,169 It just moves the frustration from data movement to people. 78 00:02:29,169 --> 00:02:31,729 A SQL developer needs a SQL experience because that's how 79 00:02:31,729 --> 00:02:33,129 they build and query data. 80 00:02:33,129 --> 00:02:34,729 A data engineer needs spark for 81 00:02:34,729 --> 00:02:35,930 large scale processing and 82 00:02:35,930 --> 00:02:36,770 notebooks. 83 00:02:36,770 --> 00:02:37,969 Analysts need reports that 84 00:02:37,969 --> 00:02:38,889 answer business questions 85 00:02:38,889 --> 00:02:40,129 without inspecting files or 86 00:02:40,129 --> 00:02:41,169 writing code. 87 00:02:41,169 --> 00:02:42,490 Applications need dependable 88 00:02:42,490 --> 00:02:43,569 access in the format they 89 00:02:43,569 --> 00:02:44,330 expect. 90 00:02:44,330 --> 00:02:45,449 The experience can differ 91 00:02:45,449 --> 00:02:46,569 without creating another 92 00:02:46,569 --> 00:02:47,650 physical copy and that 93 00:02:47,650 --> 00:02:48,490 separation matters. 94 00:02:48,490 --> 00:02:51,930 Storage answers the question where does the data live? 95 00:02:51,930 --> 00:02:55,009 The tool answers how does this person work with it? 96 00:02:55,009 --> 00:02:56,050 Once we separate those two 97 00:02:56,050 --> 00:02:57,129 questions, the mess starts 98 00:02:57,129 --> 00:02:58,050 looking fixable. 99 00:02:58,050 --> 00:02:59,400 But a single shared place can 100 00:02:59,400 --> 00:03:00,620 sound like a disaster waiting to 101 00:03:00,620 --> 00:03:01,360 happen. 102 00:03:01,360 --> 00:03:02,439 One giant folder. 103 00:03:02,439 --> 00:03:03,479 Unclear ownership. 104 00:03:03,479 --> 00:03:04,719 Everyone stepping on everyone 105 00:03:04,719 --> 00:03:06,240 else's work fabric addresses 106 00:03:06,240 --> 00:03:07,780 that concern with structure, so 107 00:03:07,780 --> 00:03:09,000 teams can share a foundation 108 00:03:09,000 --> 00:03:09,919 without giving up their 109 00:03:09,919 --> 00:03:11,240 boundaries. 110 00:03:11,240 --> 00:03:13,039 One lake, one logical lake. 111 00:03:13,039 --> 00:03:16,240 Many teams you might think the hardest part of data storage is 112 00:03:16,240 --> 00:03:18,759 just finding a place to put everything, but the real tension 113 00:03:18,759 --> 00:03:21,319 is that every team needs their own space while still sharing 114 00:03:21,319 --> 00:03:22,120 the same numbers. 115 00:03:22,120 --> 00:03:23,439 One lake solves that by giving 116 00:03:23,439 --> 00:03:24,719 your whole organization one 117 00:03:24,719 --> 00:03:26,039 logical data lake inside 118 00:03:26,039 --> 00:03:27,240 Microsoft fabric. 119 00:03:27,240 --> 00:03:29,240 The word logical matters here. 120 00:03:29,240 --> 00:03:31,800 You're not looking at one physical hard drive sitting in a 121 00:03:31,800 --> 00:03:32,680 single server room. 122 00:03:32,680 --> 00:03:35,199 Instead, you get one connected view of all your organizational 123 00:03:35,199 --> 00:03:38,439 data inside fabric spread across accounts and departments, but 124 00:03:38,439 --> 00:03:40,039 looking like a single lake. 125 00:03:40,039 --> 00:03:41,960 And here's the part that catches people off guard. 126 00:03:41,960 --> 00:03:43,360 When fabric is available in your 127 00:03:43,360 --> 00:03:44,759 tenant, one lake is already 128 00:03:44,759 --> 00:03:45,240 there. 129 00:03:45,240 --> 00:03:46,719 You don't provision a storage account. 130 00:03:46,719 --> 00:03:48,159 You don't create a lake, choose 131 00:03:48,159 --> 00:03:49,639 its infrastructure, or ask 132 00:03:49,639 --> 00:03:50,960 someone to maintain the base 133 00:03:50,960 --> 00:03:51,800 platform. 134 00:03:51,800 --> 00:03:54,039 Each fabric tenant has exactly one one lake. 135 00:03:54,039 --> 00:03:55,280 That simple rule removes a 136 00:03:55,280 --> 00:03:56,240 surprising amount of early 137 00:03:56,240 --> 00:03:56,759 architecture. 138 00:03:56,759 --> 00:03:59,360 Debate Teams spend less time asking where the next lake 139 00:03:59,360 --> 00:04:02,300 should live because the shared storage layer already exists. 140 00:04:02,300 --> 00:04:05,259 The question shifts from where do we put it to how do we 141 00:04:05,259 --> 00:04:07,419 organize and govern the data that enters it? 142 00:04:07,419 --> 00:04:09,099 Microsoft often compares one leg 143 00:04:09,099 --> 00:04:10,580 to OneDrive for data, and the 144 00:04:10,580 --> 00:04:11,939 comparison helps if you take it 145 00:04:11,939 --> 00:04:12,740 literally. 146 00:04:12,740 --> 00:04:13,939 OneDrive doesn't force every 147 00:04:13,939 --> 00:04:15,419 department into one document or 148 00:04:15,419 --> 00:04:16,420 one folder. 149 00:04:16,420 --> 00:04:18,420 It gives the organization a common service. 150 00:04:18,420 --> 00:04:19,420 While people still work in 151 00:04:19,420 --> 00:04:21,100 spaces they can understand and 152 00:04:21,100 --> 00:04:22,699 control. 153 00:04:22,699 --> 00:04:25,379 One lake follows that same file and folder model. 154 00:04:25,379 --> 00:04:27,779 Under the tenant boundary, you organize data through domains, 155 00:04:27,779 --> 00:04:30,620 then workspaces, then fabric items, then the files and tables 156 00:04:30,620 --> 00:04:31,779 inside those items. 157 00:04:31,779 --> 00:04:32,939 Each level gives people a 158 00:04:32,939 --> 00:04:34,379 clearer place to look and a 159 00:04:34,379 --> 00:04:35,220 clearer boundary for 160 00:04:35,220 --> 00:04:36,819 responsibility. 161 00:04:36,819 --> 00:04:39,420 A domain groups work around a business area like finance, 162 00:04:39,420 --> 00:04:41,100 operations, or product. 163 00:04:41,100 --> 00:04:42,139 Within that domain. 164 00:04:42,139 --> 00:04:43,699 Workspaces let teams create and 165 00:04:43,699 --> 00:04:44,939 manage the fabric items they 166 00:04:44,939 --> 00:04:45,420 need. 167 00:04:45,420 --> 00:04:47,660 Those items might include a lake house or a warehouse, and 168 00:04:47,660 --> 00:04:49,740 beneath them sit the actual files and tables. 169 00:04:49,740 --> 00:04:51,899 Think about what that changes for a finance team. 170 00:04:51,899 --> 00:04:54,779 Finance owns its workspace managers who can enter it and 171 00:04:54,779 --> 00:04:57,490 takes responsibility for the data products it publishes. 172 00:04:57,490 --> 00:04:59,930 Operations doesn't need to take over that workspace just to use 173 00:04:59,930 --> 00:05:02,569 the same overall storage foundation product, keeps its 174 00:05:02,569 --> 00:05:04,970 own workspace rules and day to day work as well. 175 00:05:04,970 --> 00:05:06,050 So this isn't centralized 176 00:05:06,050 --> 00:05:07,009 control pretending to be 177 00:05:07,009 --> 00:05:07,850 collaboration. 178 00:05:07,850 --> 00:05:09,050 It's distributed ownership 179 00:05:09,050 --> 00:05:10,490 inside a shared organizational 180 00:05:10,490 --> 00:05:11,129 boundary. 181 00:05:11,129 --> 00:05:13,529 The tenant administrator controls the outer boundary, but 182 00:05:13,529 --> 00:05:16,110 every team doesn't need to wait for one central data group to 183 00:05:16,110 --> 00:05:19,209 create every folder, table, or item that matters. 184 00:05:19,209 --> 00:05:20,410 Because data ownership works 185 00:05:20,410 --> 00:05:21,610 best close to the people who 186 00:05:21,610 --> 00:05:22,889 understand the business process 187 00:05:22,889 --> 00:05:23,569 behind it. 188 00:05:23,569 --> 00:05:26,410 Finance knows what an approved revenue number means. 189 00:05:26,410 --> 00:05:29,569 Operations knows how inventory status changes during the day. 190 00:05:29,569 --> 00:05:32,730 Product knows which customer events belong in a usage table. 191 00:05:32,730 --> 00:05:35,850 One Lake gives those teams room to own their part without 192 00:05:35,850 --> 00:05:37,610 isolating it from everyone else. 193 00:05:37,610 --> 00:05:39,529 Inside a workspace, fabric items 194 00:05:39,529 --> 00:05:40,970 provide the experience that fits 195 00:05:40,970 --> 00:05:41,769 the job. 196 00:05:41,769 --> 00:05:43,129 A lake house gives engineers a 197 00:05:43,129 --> 00:05:44,370 place to work with files and 198 00:05:44,370 --> 00:05:45,170 tables. 199 00:05:45,170 --> 00:05:46,810 A warehouse gives SQL focused 200 00:05:46,810 --> 00:05:48,209 teams, schemas and tables in a 201 00:05:48,209 --> 00:05:49,370 familiar environment. 202 00:05:49,370 --> 00:05:51,370 They feel different because the work is different. 203 00:05:51,370 --> 00:05:53,129 But here's the head fake underneath. 204 00:05:53,129 --> 00:05:54,170 Both connect to one lake. 205 00:05:54,170 --> 00:05:55,250 You aren't choosing between a 206 00:05:55,250 --> 00:05:56,360 lake lakehouse in one storage 207 00:05:56,360 --> 00:05:57,480 world and a warehouse somewhere 208 00:05:57,480 --> 00:05:57,879 else. 209 00:05:57,879 --> 00:06:00,160 You're choosing the way people interact with data stored in the 210 00:06:00,160 --> 00:06:01,360 same organizational lake. 211 00:06:01,360 --> 00:06:02,839 That's the part many people miss. 212 00:06:02,839 --> 00:06:05,319 One lake doesn't demand that every person work the same way. 213 00:06:05,319 --> 00:06:07,959 It creates a common storage layer, while fabric gives each 214 00:06:07,959 --> 00:06:10,519 team an interface that matches its skills and needs. 215 00:06:10,519 --> 00:06:12,480 But shared storage alone won't solve much. 216 00:06:12,480 --> 00:06:15,439 If each engine needs its own converted version before it can 217 00:06:15,439 --> 00:06:16,399 read the data. 218 00:06:16,399 --> 00:06:19,980 The next question is how one stored data set can work for SQL 219 00:06:19,980 --> 00:06:23,720 Spark and Power BI without starting another copy cycle. 220 00:06:23,720 --> 00:06:25,759 One copy multiple ways to work. 221 00:06:25,759 --> 00:06:27,600 One copy of data can sound restrictive. 222 00:06:27,600 --> 00:06:30,240 At first, you might hear it and think one team owns the table, 223 00:06:30,240 --> 00:06:32,879 one tool reads it, and everyone else has to wait in line. 224 00:06:32,879 --> 00:06:34,000 That isn't the model. 225 00:06:34,000 --> 00:06:35,660 One copy means one stored data 226 00:06:35,660 --> 00:06:37,279 set can support many kinds of 227 00:06:37,279 --> 00:06:39,120 work, while the teams using it 228 00:06:39,120 --> 00:06:40,399 keep the tools that fit their 229 00:06:40,399 --> 00:06:41,040 jobs. 230 00:06:41,040 --> 00:06:43,600 The shared table sits beneath fabric items in an open format 231 00:06:43,600 --> 00:06:45,079 called Delta parquet. 232 00:06:45,079 --> 00:06:47,959 Delta parquet lets different engines recognize the same table 233 00:06:47,959 --> 00:06:51,019 without each team exporting it, converting it, and storing 234 00:06:51,019 --> 00:06:52,480 another version somewhere else. 235 00:06:52,480 --> 00:06:54,000 The data stays in place, but the 236 00:06:54,000 --> 00:06:55,180 way people work with it can 237 00:06:55,180 --> 00:06:55,699 change. 238 00:06:55,699 --> 00:06:56,959 For a civil engineer, that means 239 00:06:56,959 --> 00:06:58,180 a warehouse can still feel like 240 00:06:58,180 --> 00:06:58,660 a warehouse. 241 00:06:58,660 --> 00:06:59,899 They use t-SQL. 242 00:06:59,899 --> 00:07:01,459 Work with schemas and tables, 243 00:07:01,459 --> 00:07:02,980 load data, and run the queries 244 00:07:02,980 --> 00:07:04,540 they already know behind that 245 00:07:04,540 --> 00:07:05,740 familiar experience. 246 00:07:05,740 --> 00:07:07,379 The table lives in one lake. 247 00:07:07,379 --> 00:07:08,540 The SQL team doesn't need to 248 00:07:08,540 --> 00:07:10,120 turn into a spark team just to 249 00:07:10,120 --> 00:07:11,220 take part in a shared data 250 00:07:11,220 --> 00:07:11,779 model. 251 00:07:11,779 --> 00:07:14,420 At the same time, a data engineer opens that same data 252 00:07:14,420 --> 00:07:15,660 through Apache Spark. 253 00:07:15,660 --> 00:07:17,060 Spark gives engineers and data 254 00:07:17,060 --> 00:07:18,259 scientists the tools they need 255 00:07:18,259 --> 00:07:19,660 for larger transformations, 256 00:07:19,660 --> 00:07:20,740 notebooks, and open source 257 00:07:20,740 --> 00:07:21,379 libraries. 258 00:07:21,379 --> 00:07:24,459 They read the Delta table, write changes back in the same format 259 00:07:24,459 --> 00:07:26,980 and hand the result to the next person without another export 260 00:07:26,980 --> 00:07:28,220 job sitting in the middle. 261 00:07:28,220 --> 00:07:30,779 That sounds small until you look at the usual alternative. 262 00:07:30,779 --> 00:07:32,779 A spark process creates a transform table. 263 00:07:32,779 --> 00:07:34,540 Someone copies it into a SQL store. 264 00:07:34,540 --> 00:07:36,740 Then an analyst imports it into power BI. 265 00:07:36,740 --> 00:07:39,100 Each handoff creates a delay and a new place where the data can 266 00:07:39,100 --> 00:07:40,500 drift away from the source. 267 00:07:40,500 --> 00:07:43,819 With one lake, the power BI side changes as well. 268 00:07:43,819 --> 00:07:47,300 A semantic model uses direct lake to access data in one lake 269 00:07:47,300 --> 00:07:49,980 without creating a separate import copy for the business 270 00:07:49,980 --> 00:07:52,459 user, the report still feels fast and familiar. 271 00:07:52,459 --> 00:07:53,889 They open a dashboard filter 272 00:07:53,889 --> 00:07:55,689 sales by region and ask why a 273 00:07:55,689 --> 00:07:56,569 number changed. 274 00:07:56,569 --> 00:07:59,410 They don't need to know whether spark or SQL prepared the table. 275 00:07:59,410 --> 00:08:00,689 The real tension here is that 276 00:08:00,689 --> 00:08:02,269 teams often mistake flexibility 277 00:08:02,269 --> 00:08:03,569 for duplication. 278 00:08:03,569 --> 00:08:06,009 They assume every engine needs its own stored version of the 279 00:08:06,009 --> 00:08:09,129 data, because that was often how platforms worked in the past. 280 00:08:09,129 --> 00:08:11,410 Fabric separates the engine from the stored table. 281 00:08:11,410 --> 00:08:12,490 You choose the engine for the 282 00:08:12,490 --> 00:08:13,569 task while preserving one 283 00:08:13,569 --> 00:08:14,529 approved source. 284 00:08:14,529 --> 00:08:16,850 Picture a sales data set arriving from a source system. 285 00:08:16,850 --> 00:08:18,290 It lands once in one lake. 286 00:08:18,290 --> 00:08:21,449 A data engineer uses spark to clean customer IDs, handle 287 00:08:21,449 --> 00:08:24,009 missing values, and prepare the table for analysis. 288 00:08:24,009 --> 00:08:25,329 The warehouse team then queries 289 00:08:25,329 --> 00:08:26,490 that same prepared table with 290 00:08:26,490 --> 00:08:27,970 t-SQL to answer detailed sales 291 00:08:27,970 --> 00:08:28,730 questions. 292 00:08:28,730 --> 00:08:31,670 Power BI reads it through a semantic model using direct Lake 293 00:08:31,670 --> 00:08:33,850 and leadership sees the results in a report. 294 00:08:33,850 --> 00:08:37,009 One data set different work no chain of copies passing from 295 00:08:37,009 --> 00:08:37,850 team to team. 296 00:08:37,850 --> 00:08:39,090 That doesn't mean every person 297 00:08:39,090 --> 00:08:40,210 can change the table whenever 298 00:08:40,210 --> 00:08:40,970 they want. 299 00:08:40,970 --> 00:08:42,529 One copy doesn't remove ownership. 300 00:08:42,529 --> 00:08:43,929 It doesn't remove data quality 301 00:08:43,929 --> 00:08:45,169 checks, approval steps, or 302 00:08:45,169 --> 00:08:46,090 access rules. 303 00:08:46,090 --> 00:08:48,570 In fact, those things become more important when more people 304 00:08:48,570 --> 00:08:50,169 depend on the same source. 305 00:08:50,169 --> 00:08:52,830 Someone still needs to define what a sales record means. 306 00:08:52,830 --> 00:08:55,230 someone needs to decide how canceled orders appear. 307 00:08:55,230 --> 00:08:58,029 Someone needs to check whether the load completed and whether 308 00:08:58,029 --> 00:09:00,309 the table is fit for reporting. 309 00:09:00,309 --> 00:09:03,230 Access still needs to match the job, so a person who can view a 310 00:09:03,230 --> 00:09:05,789 report doesn't automatically gain permission to alter the 311 00:09:05,789 --> 00:09:06,669 data behind it. 312 00:09:06,669 --> 00:09:09,990 One lake reduces duplication, it doesn't remove responsibility, 313 00:09:09,990 --> 00:09:11,950 and there's another limit worth keeping in view. 314 00:09:11,950 --> 00:09:13,309 One stored copy inside fabric 315 00:09:13,309 --> 00:09:14,429 helps a lot, but most 316 00:09:14,429 --> 00:09:15,950 organizations already keep data 317 00:09:15,950 --> 00:09:17,149 and other systems, clouds, and 318 00:09:17,149 --> 00:09:18,190 business platforms. 319 00:09:18,190 --> 00:09:21,230 Copying all of that data into fabric just to make it visible 320 00:09:21,230 --> 00:09:23,389 would create the same problem in a new place. 321 00:09:23,389 --> 00:09:26,190 Shortcuts data where it lives. 322 00:09:26,190 --> 00:09:29,309 A shortcut solves the problem of data that shouldn't move at all. 323 00:09:29,309 --> 00:09:30,230 Think about it. 324 00:09:30,230 --> 00:09:31,429 Data that belongs to sales 325 00:09:31,429 --> 00:09:32,269 should stay under sales 326 00:09:32,269 --> 00:09:33,950 ownership, but finance needs it 327 00:09:33,950 --> 00:09:34,669 to. 328 00:09:34,669 --> 00:09:37,029 The instinct says to copy it, creating another copy and 329 00:09:37,029 --> 00:09:38,309 another refresh schedule. 330 00:09:38,309 --> 00:09:39,269 But here's the head fake. 331 00:09:39,269 --> 00:09:40,389 You don't need a copy. 332 00:09:40,389 --> 00:09:43,750 A shortcut is a virtual reference to a file, folder or 333 00:09:43,750 --> 00:09:45,350 table in another location. 334 00:09:45,350 --> 00:09:47,669 It appears right where your team needs to work while the data 335 00:09:47,669 --> 00:09:48,789 stays at its source. 336 00:09:48,789 --> 00:09:50,389 Fabric just gives you a path to it. 337 00:09:50,389 --> 00:09:51,870 Take finance as an example. 338 00:09:51,870 --> 00:09:54,419 The sales team owns an approved sales table in its own 339 00:09:54,419 --> 00:09:57,379 workspace, with all the rules and checks that turn raw orders 340 00:09:57,379 --> 00:09:58,940 into accepted numbers. 341 00:09:58,940 --> 00:10:01,779 Finance needs that table for forecasting, but rebuilding the 342 00:10:01,779 --> 00:10:04,740 sales pipeline inside a finance workspace means another copy and 343 00:10:04,740 --> 00:10:06,100 another schedule to maintain. 344 00:10:06,100 --> 00:10:09,460 Instead, finance creates a shortcut to that approved table. 345 00:10:09,460 --> 00:10:11,659 From their point of view, the table appears in their 346 00:10:11,659 --> 00:10:14,580 workspace, ready to combine with budgets and forecasts. 347 00:10:14,580 --> 00:10:15,899 Sales still owns the source 348 00:10:15,899 --> 00:10:17,500 table, and nobody has to build a 349 00:10:17,500 --> 00:10:18,779 second pipeline just to move 350 00:10:18,779 --> 00:10:20,139 sales data across an internal 351 00:10:20,139 --> 00:10:20,580 boundary. 352 00:10:20,580 --> 00:10:21,940 That is an internal shortcut. 353 00:10:21,940 --> 00:10:23,460 It lets one workspace reuse a 354 00:10:23,460 --> 00:10:24,620 table or folder from another 355 00:10:24,620 --> 00:10:25,419 workspace. 356 00:10:25,419 --> 00:10:26,779 Even when teams sit in different 357 00:10:26,779 --> 00:10:28,700 business domains, the data stays 358 00:10:28,700 --> 00:10:30,059 under the ownership of the team 359 00:10:30,059 --> 00:10:31,460 that understands it, while other 360 00:10:31,460 --> 00:10:32,940 teams consume it where it fits 361 00:10:32,940 --> 00:10:33,580 their work. 362 00:10:33,580 --> 00:10:35,460 Shortcuts also reach beyond fabric. 363 00:10:35,460 --> 00:10:38,620 A team can create references to data in Azure Data Lake Storage, 364 00:10:38,620 --> 00:10:40,259 Amazon S3, or Dataverse. 365 00:10:40,259 --> 00:10:43,019 So if customer data already sits in Dataverse or an existing data 366 00:10:43,019 --> 00:10:46,620 platform stores files in ADLs or S3, you don't automatically need 367 00:10:46,620 --> 00:10:48,500 a copy in one lake before people can work with it. 368 00:10:48,500 --> 00:10:49,539 You might think that means one 369 00:10:49,539 --> 00:10:51,009 leg pulls every outside system 370 00:10:51,009 --> 00:10:51,639 into fabric. 371 00:10:51,639 --> 00:10:52,279 It doesn't. 372 00:10:52,279 --> 00:10:53,720 The source stays where it lives. 373 00:10:53,720 --> 00:10:54,960 The shortcut makes that source 374 00:10:54,960 --> 00:10:56,639 visible through one lake, which 375 00:10:56,639 --> 00:10:57,919 reduces the pressure to move 376 00:10:57,919 --> 00:10:59,399 data simply because another team 377 00:10:59,399 --> 00:11:00,039 needs it. 378 00:11:00,039 --> 00:11:01,399 That changes the conversation 379 00:11:01,399 --> 00:11:02,759 instead of when can we copy this 380 00:11:02,759 --> 00:11:03,440 table? 381 00:11:03,440 --> 00:11:06,159 The question becomes, can we safely use the approved source 382 00:11:06,159 --> 00:11:07,399 where it already lives? 383 00:11:08,399 --> 00:11:09,600 Fewer copies mean fewer 384 00:11:09,600 --> 00:11:11,080 scheduled loads, fewer failed 385 00:11:11,080 --> 00:11:12,559 refresh chains, and fewer 386 00:11:12,559 --> 00:11:13,879 moments where two teams look at 387 00:11:13,879 --> 00:11:15,039 two versions of the same sales 388 00:11:15,039 --> 00:11:15,759 number. 389 00:11:15,759 --> 00:11:17,320 But here's where the twist comes in. 390 00:11:17,320 --> 00:11:19,519 A shortcut isn't a magic permission pass. 391 00:11:19,519 --> 00:11:21,659 Creating one doesn't give someone blanket access to 392 00:11:21,659 --> 00:11:22,879 everything behind it. 393 00:11:22,879 --> 00:11:24,159 Access rules still matter, and 394 00:11:24,159 --> 00:11:26,000 people only see or use data 395 00:11:26,000 --> 00:11:27,440 where their permissions allow 396 00:11:27,440 --> 00:11:27,759 it. 397 00:11:27,759 --> 00:11:29,000 That boundary matters because 398 00:11:29,000 --> 00:11:30,879 shared access should expand the 399 00:11:30,879 --> 00:11:32,720 use of trusted data, not quietly 400 00:11:32,720 --> 00:11:33,879 open sensitive data to the 401 00:11:33,879 --> 00:11:35,159 entire organization. 402 00:11:35,159 --> 00:11:37,039 The shortcut also depends on the source. 403 00:11:37,039 --> 00:11:39,799 If the source data arrives late, changes structure without 404 00:11:39,799 --> 00:11:42,960 warning or contains poor quality records, the shortcut won't fix 405 00:11:42,960 --> 00:11:43,600 any of that. 406 00:11:43,600 --> 00:11:46,559 It removes unnecessary movement, but it can't replace an owner 407 00:11:46,559 --> 00:11:49,279 who defines the table, monitors its freshness, and tells 408 00:11:49,279 --> 00:11:52,509 consumers what they can trust, so use shortcuts to avoid 409 00:11:52,509 --> 00:11:53,990 building copies by default. 410 00:11:53,990 --> 00:11:55,730 Then confirm who owns the source 411 00:11:55,730 --> 00:11:57,350 who can access it, what happens 412 00:11:57,350 --> 00:11:58,350 when it changes, and how 413 00:11:58,350 --> 00:11:59,710 consumers know whether the data 414 00:11:59,710 --> 00:12:00,669 is current. 415 00:12:00,669 --> 00:12:01,870 A shortcut gives you a cleaner 416 00:12:01,870 --> 00:12:03,470 path to the data, but it doesn't 417 00:12:03,470 --> 00:12:04,750 remove the need to manage that 418 00:12:04,750 --> 00:12:05,389 path. 419 00:12:05,389 --> 00:12:08,090 Shared access only helps when people can find the right data, 420 00:12:08,090 --> 00:12:10,909 understand what it means, and control who can use it. 421 00:12:10,909 --> 00:12:12,830 How fabric makes the lake usable. 422 00:12:12,830 --> 00:12:14,549 A shared data lake only works if 423 00:12:14,549 --> 00:12:15,750 people can actually work with 424 00:12:15,750 --> 00:12:15,990 it. 425 00:12:15,990 --> 00:12:18,470 The big story here is that fabric handles that by giving 426 00:12:18,470 --> 00:12:21,389 different teams familiar ways to interact with the same storage 427 00:12:21,389 --> 00:12:23,830 layer, rather than asking everyone to become a data 428 00:12:23,830 --> 00:12:25,990 engineer for engineering work. 429 00:12:25,990 --> 00:12:29,909 A lakehouse gives you two areas files for raw material like CSV, 430 00:12:29,909 --> 00:12:34,070 Json images or logs, and tables for delta tables that spark can 431 00:12:34,070 --> 00:12:34,990 read and write. 432 00:12:34,990 --> 00:12:36,509 An engineer can land raw data, 433 00:12:36,509 --> 00:12:37,629 clean it, and turn it into 434 00:12:37,629 --> 00:12:38,789 tables without leaving the 435 00:12:38,789 --> 00:12:40,149 lakehouse experience. 436 00:12:40,149 --> 00:12:41,870 SQL teams see a different view. 437 00:12:41,870 --> 00:12:43,110 A fabric warehouse gives them 438 00:12:43,110 --> 00:12:45,149 schemas, tables, and t-SQL so 439 00:12:45,149 --> 00:12:46,710 they can query and manage data 440 00:12:46,710 --> 00:12:47,549 in the way they already 441 00:12:47,549 --> 00:12:48,429 understand. 442 00:12:48,429 --> 00:12:49,700 The warehouse feels familiar, 443 00:12:49,700 --> 00:12:50,980 but its data still sits in one 444 00:12:50,980 --> 00:12:51,820 lake underneath. 445 00:12:51,820 --> 00:12:54,460 You don't need a separate storage plan just because one 446 00:12:54,460 --> 00:12:57,980 team speaks SQL and another works in spark. 447 00:12:57,980 --> 00:12:59,139 Here's the head fake. 448 00:12:59,139 --> 00:13:00,539 A lake house and a warehouse 449 00:13:00,539 --> 00:13:01,740 aren't competing storage 450 00:13:01,740 --> 00:13:02,500 islands. 451 00:13:02,500 --> 00:13:05,220 They are different working surfaces over the same data. 452 00:13:05,220 --> 00:13:06,940 Choose the surface that fits the job. 453 00:13:06,940 --> 00:13:08,980 Some people won't start in fabric at all. 454 00:13:08,980 --> 00:13:11,580 One Lake File Explorer Let's permitted users browse 455 00:13:11,580 --> 00:13:14,500 workspaces and data files from Windows File Explorer. 456 00:13:14,500 --> 00:13:15,580 They can move through folders 457 00:13:15,580 --> 00:13:16,659 they have access to and work 458 00:13:16,659 --> 00:13:17,740 with files in a familiar 459 00:13:17,740 --> 00:13:19,379 interface that lowers the 460 00:13:19,379 --> 00:13:20,860 barrier for users who understand 461 00:13:20,860 --> 00:13:22,580 business data but don't want to 462 00:13:22,580 --> 00:13:24,019 begin with a notebook or command 463 00:13:24,019 --> 00:13:24,460 line. 464 00:13:24,460 --> 00:13:27,340 Fabric also stays open to tools outside its own interface. 465 00:13:27,340 --> 00:13:29,039 One lake supports the same ADLs 466 00:13:29,039 --> 00:13:31,379 Gen2 APIs and APFs paths used by 467 00:13:31,379 --> 00:13:33,039 compatible tools, including 468 00:13:33,039 --> 00:13:34,179 Azure Databricks. 469 00:13:34,179 --> 00:13:36,860 If your team already has a process that can read and write 470 00:13:36,860 --> 00:13:40,019 through those paths, it can connect to one leg rather than 471 00:13:40,019 --> 00:13:43,019 forcing another export into a separate storage location. 472 00:13:43,019 --> 00:13:45,139 Access doesn't mean everyone should see everything. 473 00:13:45,139 --> 00:13:47,340 Finding the right data set starts with structure. 474 00:13:47,340 --> 00:13:47,980 Workspaces. 475 00:13:47,980 --> 00:13:50,960 Group A teams work domains group work around business areas. 476 00:13:50,960 --> 00:13:51,759 Item permissions. 477 00:13:51,759 --> 00:13:53,039 Narrow access to the specific 478 00:13:53,039 --> 00:13:54,600 data, product or tool a person 479 00:13:54,600 --> 00:13:55,120 needs. 480 00:13:55,120 --> 00:13:56,840 Data catalog views help people 481 00:13:56,840 --> 00:13:58,360 search across available items 482 00:13:58,360 --> 00:13:59,840 and understand what they are 483 00:13:59,840 --> 00:14:01,279 looking at before they build 484 00:14:01,279 --> 00:14:02,039 something new. 485 00:14:02,039 --> 00:14:03,679 Keep access narrow by default. 486 00:14:03,679 --> 00:14:07,019 If an analyst only needs an approved table, grant access to 487 00:14:07,019 --> 00:14:08,159 that table or item. 488 00:14:08,159 --> 00:14:11,360 Rather than handing over broad workspace access. 489 00:14:11,360 --> 00:14:13,440 A broad role can feel faster in 490 00:14:13,440 --> 00:14:14,759 the moment, but it often creates 491 00:14:14,759 --> 00:14:16,600 clean up work later and makes it 492 00:14:16,600 --> 00:14:17,960 harder to see who can reach 493 00:14:17,960 --> 00:14:18,960 sensitive data. 494 00:14:18,960 --> 00:14:20,320 One more separation helps. 495 00:14:20,320 --> 00:14:21,960 One lake holds the data. 496 00:14:21,960 --> 00:14:24,519 Fabric capacities provide the compute that runs workloads 497 00:14:24,519 --> 00:14:28,059 against it a capacity, power, spark jobs, SQL activity 498 00:14:28,059 --> 00:14:31,080 reports, and other processing, storage, and Compute Connect. 499 00:14:31,080 --> 00:14:32,919 But they aren't the same decision. 500 00:14:32,919 --> 00:14:34,320 You can manage workload power 501 00:14:34,320 --> 00:14:35,620 without treating every data 502 00:14:35,620 --> 00:14:36,399 store as a separate 503 00:14:36,399 --> 00:14:37,759 infrastructure project. 504 00:14:37,759 --> 00:14:40,679 The technology can cut out many unnecessary copies. 505 00:14:40,679 --> 00:14:43,320 People still need to change the habit that created those copies 506 00:14:43,320 --> 00:14:46,559 in the first place, but that's where the real opportunity lies. 507 00:14:46,559 --> 00:14:49,590 When you stop treating every data need as a new copy project, 508 00:14:49,590 --> 00:14:52,990 you free your team to focus on what the data actually means. 509 00:14:52,990 --> 00:14:55,269 Putting one copy into practice. 510 00:14:55,269 --> 00:14:56,429 The real tension here isn't 511 00:14:56,429 --> 00:14:57,750 about migrating everything at 512 00:14:57,750 --> 00:14:58,509 once. 513 00:14:58,509 --> 00:15:01,509 It's about one high use data set where duplicate pipelines are 514 00:15:01,509 --> 00:15:04,549 already causing delays, failed refreshes, and arguments over 515 00:15:04,549 --> 00:15:06,350 which report is the real one. 516 00:15:06,350 --> 00:15:09,549 Sales, customer inventory, or finance data often exposes the 517 00:15:09,549 --> 00:15:12,629 problem quickly because multiple teams are fighting over who's 518 00:15:12,629 --> 00:15:13,870 version is current. 519 00:15:13,870 --> 00:15:16,629 You might think the solution is a massive migration plan, but 520 00:15:16,629 --> 00:15:18,029 that's the head fake. 521 00:15:18,029 --> 00:15:19,629 The actual first move is much simpler. 522 00:15:19,629 --> 00:15:21,389 Pick a single data set and start there. 523 00:15:21,389 --> 00:15:23,750 But before you build a shortcut, you need an owner. 524 00:15:23,750 --> 00:15:26,389 That owner has to publish the approved source, define what it 525 00:15:26,389 --> 00:15:28,950 means and tell consumers when they can trust it. 526 00:15:28,950 --> 00:15:32,110 Without that decision, a shared table just becomes a better 527 00:15:32,110 --> 00:15:33,549 connected source of confusion. 528 00:15:33,549 --> 00:15:35,309 Now here is where it gets interesting. 529 00:15:35,309 --> 00:15:38,009 Each team can keep the tool that fits their work while shared 530 00:15:38,009 --> 00:15:40,470 tables move toward delta parquet in one lake. 531 00:15:40,470 --> 00:15:42,549 The goal isn't to standardize everyone's workflow. 532 00:15:42,549 --> 00:15:45,190 The goal is to stop creating separate stored versions. 533 00:15:45,190 --> 00:15:48,090 Just because a different team needs a different interface. 534 00:15:48,090 --> 00:15:49,250 So before a new pipeline 535 00:15:49,250 --> 00:15:50,889 appears, ask whether a shortcut 536 00:15:50,889 --> 00:15:51,929 can connect the team to the 537 00:15:51,929 --> 00:15:52,850 approved source. 538 00:15:52,850 --> 00:15:54,210 Check the access rules, who owns 539 00:15:54,210 --> 00:15:55,529 the source and how freshness is 540 00:15:55,529 --> 00:15:56,210 monitored. 541 00:15:56,210 --> 00:15:57,750 If those answers aren't clear, 542 00:15:57,750 --> 00:15:59,210 don't hide the problem behind 543 00:15:59,210 --> 00:16:00,129 another copy. 544 00:16:00,129 --> 00:16:01,529 Track the practical results. 545 00:16:01,529 --> 00:16:03,889 Count the duplicate pipelines that disappear, the stored 546 00:16:03,889 --> 00:16:06,649 copies that no longer need refreshes, and how long it takes 547 00:16:06,649 --> 00:16:08,769 a team to reach approved data. 548 00:16:08,769 --> 00:16:12,090 Listen for fewer meetings spent defending competing numbers. 549 00:16:12,090 --> 00:16:14,690 Copying data can feel safe because each team controls its 550 00:16:14,690 --> 00:16:17,289 own version, but every new version becomes another thing 551 00:16:17,289 --> 00:16:21,809 someone has to refresh, secure, explain, and eventually fix. 552 00:16:21,809 --> 00:16:22,610 Conclusion. 553 00:16:22,610 --> 00:16:23,769 Stop building copies. 554 00:16:23,769 --> 00:16:25,649 So here's what it all comes down to. 555 00:16:25,649 --> 00:16:28,769 One Lake changes the question from where should this team copy 556 00:16:28,769 --> 00:16:33,330 the data to who owns the trusted source that every team can use? 557 00:16:33,330 --> 00:16:36,289 But here's the takeaway that shift is the key to stopping the 558 00:16:36,289 --> 00:16:37,690 cycle of technical debt. 559 00:16:37,690 --> 00:16:38,769 And that's exactly why you 560 00:16:38,769 --> 00:16:40,809 should subscribe to M365 FM for 561 00:16:40,809 --> 00:16:42,250 practical ways to use fabric, 562 00:16:42,250 --> 00:16:44,090 power, platform and AI without 563 00:16:44,090 --> 00:16:45,129 building more technical debt 564 00:16:45,129 --> 00:16:46,409 into the work your teams depend 565 00:16:46,409 --> 00:16:46,929 on.