Read the full episode transcript
Generated from the exported YouTube subtitle track. Timestamps appear roughly once per minute and link back to that point on YouTube.
We are listening to the Data Hustle podcast, where data is turned into business impact. Today on the Data Hustle, we're joined by Jimmy Pang, a data and analytics leader who has spent more than eight years building teams, platforms and decision systems across companies like Foodpanda, HelloFresh, Delivery Hero and Vestiaire Collective. His work has included redesigning the data architecture behind more than 24 million euros in payments, producing fraud incidents by 80%, cutting customer service response times in half, and managing an analytics estate of more than 1700 dbt models across seven business domains. Today, Jimmy's building Data Biz, where he combines fractional data leadership, open source experimentation and refreshingly blunt writing about what it actually takes to turn data into decisions. Jimmy, welcome to the Data Hustle. So first of all, I would just like to ask you, you've come all the way from Hong Kong to Germany to do data and data
analytics. I haven't heard many people making their jobs. So can you tell us and audience a little bit about what inspired you to do this? Sure. First of all,
thanks for having me. And to answer your questions, this is definitely rare to have Hong Kong coming all the way from Asia to Berlin. And I have to say it was a little bit of coincidence and a little bit of the chance. Or you could cut fate, if you want. Basically, back in seven years ago, it was a golden year for startups, especially in Europe. And basically, the cash was basically free and you have tons of startups all over the world, especially in Berlin. And there was a pretty nice nickname of Berlin Gods back in the days. It was known as Silicon Valley in Europe. So I don't know where the heck has it come from, but I decided to trust it. And I also was working in the Foodpanda Hong Kong back in the days, which was one of the subsidiaries of the Delivery Hero. So after spending some time in the Foodpanda, and I promote myself, quote unquote, promote myself to Berlin to pursue better development in the data career, which I know for sure
I want to work on and spend the rest of my career in.
That's cool. So I mean, a bunch of things that we want to dive into here. But I guess first thing that I wanted to ask about is, so you've done a lot of work with payments, which is typically very operationally sensitive data. Now, I've seen also through my experience in healthcare, which has similar viniculars that having a team that's trained in doing analytics and changing them to kind of think about data more in an operational sense is pretty hard culturally. How have you experienced this? And how do you navigate this?
Well, to clarify a little bit, when it comes to operation, it really varies a lot depending on which exact company. And we're also talking about different companies on different continents, and different days as well, and different industries as well. So to be more specific, in Foodpanda and the Delivery Hero, they were still in the business of food delivery. And for the Delivery Hero is more a bit on the side of SaaS as well. And because it's also holding company of the other food delivery companies. So when it comes to payments, it is a very different idea to another industry, like for example, e-commerce in if it's still or maybe in a HelloFresh as well. And I could also imagine in healthcare, it would also be very different when it comes to payment. But I mean, payments, it also got for a couple of different iterations over the time I see. And for layman, which I start OS as well, payment sounds more like a finance topic, actually, at first glance,
and then you think a little bit more. And you will realize it is actually more a day to day operation topic. So it's now as a finance topic, but later it got frozen under the umbrella of operations. And honestly, I think it makes more sense to put it under the umbrella of operation as well. As for the dynamics in Asia, especially for the business of Foodpanda, surprise, surprise, their primary payment method is actually COD or cash on delivery. And honestly, this kind of payment method, it is how they put it, it creates so many loopholes and difficulty and operational challenge, especially when it comes to the cash selection. And there's so many frauds hold there as well. Because I know for sure, someone has been doing some hacking and doing attack for probably Foodpanda Malaysia and generate a lot of like effect order that uses cash on delivery as well. And this is the kind of a DDoS attack for the food delivery platforms.
So yeah, payment has evolved a lot over the years. Interesting. So
you have a lot of experience working in different cultures. Can you give us some example? How do you notice like different cultures that go working with data differently or just engineering as well?
This is a very difficult answer to answer, I would say. Let me try to think about a couple of different examples here. So to make it very explicit, I'm from Hong Kong, Asia, obviously. And I have spent some years working in the German company like Delivery Hero and HelloFresh, and I also spent the latest last four years in a French company. And this free spectrum is already give me a very, very different dynamics I have observed. So for example, in Hong Kong, everything is about solving the immediate problem. Like firefighting is always something that happens on a day to day basis. Nevertheless, it's also created a problem of scalability. So the strong suit of how Hong Kong people work is they will be very flexible and yet compliant on certain frameworks because of the historical reasons of the history of Hong Kong. And yet, because of the Chinese influence, we actually don't have the culture of, for example, writing things down.
Like, for example, if we have meetings just now, we don't leave any meeting notes at all. And that would create a problem for like longer term project for people because people start forgetting about the details and what was agreed on and what was the spec of certain data stuff. And now, go to the German working culture, what I have observed, especially in HelloFresh, it was a German-German company. In comparison, Delivery Hero was not so German. It's more like international or American style, but still by nature, it's still a German company. Anyway, the point is, if it is a German company, what you could observe, for example, a meeting, 30 minutes of meetings, what would happen one week before the meetings, you would receive a pre-read and agenda and you would know exactly what to expect and what topic would be covered in this 30 minutes of meeting.
And then around five to one minutes before the meeting, people start showing up and they start having some very quick small talk. And at the time, people actually start the official topic and they will start to go through the agenda and as they would cover the items by items, there won't be shifting and there won't be any flexibility. Whatever is put on the agenda, it is there. For data and engineering and tech stuff, it is actually amazing because we know exactly what to expect. Once the spec is fixed there, it doesn't change. Now, of course, there will also be a problem because there are certain things you don't have enough time to do on discovery of the data and you find unexpected behavior or unexpected schema or even you don't even know what exact schema to expect from data at the beginning and yet you have a conversation there. And then, if you adjust on the fly, the German culture is actually not very responsive to address these kind of
challenges. And they could go very challenging if it goes to the scale like HelloFresh. And also, I need to highlight one more dimension in the German working culture. It's very, how they put it, very, very flat. As in, you don't have a very high power distance with a direct report. You basically talk directly to your boss and you could openly and directly disagree with your boss. And in French culture and Chinese culture, it could be even worse. So in French culture, you usually don't disagree with your boss and even if you do, you do it in private. And in Asian culture, especially in mainland China, you don't do that. I mean, any idea from your big boss is always big boss, and you don't dare to raise any second voice there.
And if you do any disagreement, you will be solved as a problem. So that being said, that kind of working culture, of course, if you want to push certain tech or data project, the benefit will be it goes super fast. And the drawback is also very clear. Whatever is not being prioritized by the big boss, they does not receive any attention or any care. And it's like people forget they had this project at all. So they would be completely lack of maintenance. And if it breaks, no one cares. Unless it jeopardizes the primary focus of the directive from the big boss, then it becomes in the part of our agenda again. French, I would say, is a mix of both things.
It's more like something in between, between the German working culture and Asian slash Chinese working culture. But what is very different there is about the exercise of work-life balance, especially during summer. And I mean, it wasn't that obvious for me when it comes to the summer holiday, the vacation, the biggest form. I mean, people take turns to have breaks and have vacation and so on in Berlin. But in French, in the French company, it is basically sacred. You do not dare to borrow them during those vacation days. And by the way, they also have a shorter working weeks as well. As far as I remember, they have 35 working hours per week, while German standard working hours is 40. At least it's the number of contracts. So yeah, for French, there's also one thing I need to highlight here is the idea of central.
So if we look at the history of Germany and also with France, and France is a rather simple history to understand, because to them, there's always one central. It's always Paris. There's no other middle ground. There's no other center. But over the history of Germany, there has been multiple centers and a capital due to the history and given the history of Germans. So there used to be, was it Munich? Or sometimes it was Dusseldorf and sometimes it was Berlin. So over the years, over the decades in the history of Germany, the Germans are very used to the idea of having multiple centers. And I believe this is also where the flat hierarchy comes from for them.
And it is also very realistically reflecting in the architecture of the designs, because I have found the idea of a data mesh. And there's something I got exposed to for the first time in Delivery Hero. And then HelloFresh also adopted that as well in a very German way. And I would say they did it pretty much poorly, because they just went out and dedicated all the ownership and production for the data for different teams. And yet they did not have good enough framework and also the platform as well. So it got so super chaotic. And it's basically a non-worker board for data. I still remember for one data problem, usually I will have six or seven ways to do the exact same thing. And I think it is what it means by being resourceful working as data. But the problem is in HelloFresh, I bumped into a bottleneck problem.
I tried around 11 or 13 different ways. I still cannot track it outside. Okay, that's it. I'm out of here. I mean, it's just impossible to proceed with a normal day-to-day analytics work in HelloFresh. And it's because of the separate decision making and everyone's they have their own say, and they cannot agree with each other. Even if you want to have a better pipeline for the cloud call, and you want to improve the performance of a data warehouse. By the way, they were using Impala back in the days. It basically takes months, if not years, to get there. While if it is Hong Kong, it would be done within weeks. But probably not scalable though. It would solve the immediate problem, but it's not scalable. And going back to French, French is still used to the idea of a center. That's why when I was introducing a mesh and data mesh idea, it was not actually fully federal per se. That's still a very strong idea of central. Because I had to introduce the idea of a core CDM and
analytics data mesh or ADM in short. So long story short, the CDM is the data mesh is still being owned and maintained by the central data team, which was the team I was in. And for the ADM, they were the data mesh owned by the analyst team. And there are a couple of different rules and governance rules for ADM as well. For example, ADM could only depend directly on the CDM. They should not be directly consummated from staging layer or intermediate layer, etc. And we considered the data mesh idea, but we realized the French working culture just does not allow this type of implementation. They're still expecting there would be a central team that tells us, okay, this is okay, and this is good, and this is bad, and there will be a central governance body there. So yeah, this is how we had to adopt for the implementation of the data mesh on the different culture there. That's
super interesting. I never consider to think about the cultural aspect when it comes to your data governance strategy and data mesh. But now that you put it that way, that makes a lot of sense. Actually, I
would say this is the first thing we should consider before implementing any data or tech changes. Because if I'm not mistaken, there's a law called "Conway's Law Law". It's whatever in the tech and in the data, they would reflect the people behind them. Or using a more CTO language there, you ship your organization. So you have to think about the culture, the people element and the process first, then you worry about the technology or tools. Our old school MBA or old school VBA technology here, there will be PPT, people, process tool. All but don't.
Yeah, you've told me before that you think the future is in Asia. So just for anyone listening in that is thinking of running their own data business, being a consultant, why is Asia the future? And what would you take from your experience in Germany and France and take it back to these places? Because obviously, what you mentioned, the work style is so different, it might also be just different kinds of organization, very large versus smaller, more lean organization. So how do you see those two things in front of you?
Well, to cover the first bit you mentioned here about Asia. To me, it's more pure economic code observation about the growth of GDP. And then I think of data. I mean, yes, I stuck as the data person and I also plan to throw my whole career in data. But nevertheless, I realized one very important thing is the business needs to come before anything else, before data, before AI, before tech. So about the opportunity in Asia, and by the way Dubai as well. And yeah, the opportunity I see is there are a lot of hot money, a lot of capital there. And yet I don't see a very mature tech community, especially compared to what I have firsthand exposed to Berlin, Berlin tech and data community. And probably in San Francisco and Silicon Valley as well.
So there are very big differences between the supply and demand. Like in Berlin and in Silicon Valley, there are very good supply of very good data and tech talent. And that also means it's a very competitive market for this talent as well. And on the other hand, markets like Asia, especially in Vietnam, Ho Chi Minh City, and Dubai as well, there are a lot of cash, there are a lot of investment, and yet there are not a lot of supply of this very good talent. So I would say there's economy, economic 101 here, supply and demand. Hence you go to where the money is, because there are demand of the business. And when there's a business, and then there will be tech, and then there will also be data and AI. Because the longer you work with data and tech, the more you realize all the basic foundation of tech and data and also AI as well, it requires a very capital intense investment. For example, for running AI or for running LLM, you actually need
a very strong computational power. That's why the back players in tech like Google, Microsoft, they keep building this data center. It's because they really need the strong computational power, especially in the RAM and GPU and so on. By the way, as a gamer myself, I also hate them a little bit for that, because I also make the newly released Steam Machine super expensive now. I'm getting off track here. Did that come from your original question? I think I missed a lot
of things there. I know, I know. I also felt the pain of the Steam Machine. That makes sense. That is a very expensive venture. You're talking about Google scale, but also smaller organizations. Even the salaries that it takes for a company to have a proper head of data, and then a proper person doing the data modeling, the analysis. A lot of companies get away with a team of three, but very quickly it goes through a whole football team full of people that have to do something data
related. How do I put it? For the impact of AI, what I will see is actually another classic economic code evolution over time. It's definitely not as magical as people keep pushing and keep pushing this narrative for the magical AI. AI is going to be a silver bullet to solve every problem of our own. AMS came off a realistic summer estate. To me, it is yet another boost of productivity. Yet again, going back to the foundation for economic. The whole development of human economic is all about boosting the productivity while driving down the cost. That's it. What is the primary impact of AI to us, as I foresee for the whole world, is that we just need fewer people to be able to do way more things.
Let's take finance for example, or maybe bookkeeping and accounting. If we dial back around 100 years ago, when bookkeeping was just invented, and there was a butler opening a ledger, and he was doing credit for the landlord for all these assets. How did you make it scalable? How did you do this bookkeeping for a big group of assets? If you would require around 100 to 200 bookkeepers, literally writing all the entries by hand on the physical book, and you would require four to five supervisors to manage this 100 to 200 bookkeepers. Then these four to five supervisors would be reporting to the owner of all these assets. Nowadays, we have software like MYOB, my own business, and Sage, which used to be a Peachtree. Those accounting software empowering just a classic finance team like four to five people to do what used to take 100 to 200 people. The middle-level supervisor would require to manage them. This is how I would foresee this is going to happen a lot more in
the short future as well. There will be a rise of solo founders. As in one person, only one company, there would be a lot of AI agents helping disadvantaged humans out to handle different functions in the business like marketing or finance or operations, or at most, a very, very small team, but they would be able to pull what used to take around 30 to 40 people back in without AI. What else? That would also be another big question for the employees as well. If we used the same analogy from 100 years ago, where were those 100-something to bookkeeper if they were no longer lead in the modern finance team? Where are they now? They could be going to the other company, or they become a business owner, or maybe they change their career.
We need to prepare for that kind of future instead of being swayed by the narrative being pushed by the capital market just to be pragmatic about the future from AI.
Part of me likes to think that AI is going to be able to take over a lot of our tasks, but also the easier things become, that opens up more brain capacity for us to handle other problems.
That being said, taking care of our capacity also means the execution ship. That once again emphasizes the importance of the human judgment. Once all the execution ship does it mean all of them are worth doing, I can tell you definitely not, because more things being executed, it also means more bandwidth from your human mind to manage all those executions. Yes, I would say AI actually intensifies the work. It is definitely not a magic bullet. It would make whoever is already strong at it, even stronger, and who is not used to it, they are going to have trouble adopting it.
Obviously, it is possible to just keep piling on features and complexity a lot faster now with AI. This differs a lot also between companies and cultures, as you've sort of alluded to as well, where one company or one culture might be just firefighting all the time, push everything out as fast as possible, where the on one is very careful, calculated, slow. Where do you feel is the right balance in these things?
I would say it depends. Also, for the sake of an easier discussion, I'm going to put it as an Apple approach or Google approach. Because Google, to me, at least, is always a launch first iterator. That's also explained why they have that many of kill the product over the years. As for Apple, at least for the previous couple of years, they would just make it perfect and then release one time. For AI, definitely it is empowering more for the Google model, the launch first iterator. And for business, especially when they are smaller and they need to have hyper growth, AI is definitely helping out with the speed even more, even more so. But for another approach for making it perfect and then shift once, I would say unfortunately, AI is going to work against you.
Especially if you use AI for vibe coding, for example, it actually means you are slacking off understanding or putting more thoughts of what exactly are you doing. You're just throwing one prompt to the agent and let it cope with you. And it will generate all the code changes for you. What you do as a human, you're just reviewing the code there. Honestly, I'm also good at myself as well. I don't rely for the code generated by AI. I was just looking at the critical part. It looks okay to me and the test passes in a CI by by. I will just hit the approve and merge it. Period. So that being said, it actually creates a lot more challenge in addition to the traditional technical debt. I see discussions labeling this kind of debt as a comprehension debt. I think it's a pretty good description there.
Because this is exactly what you're doing with AI. When you ask the AI to do something, you are offloading the thinking and the groundwork for the AI to do it and you are skipping apart. What is usually done by yourself or done by another human, that person would gain firsthand experience of the groundwork, no matter how time consuming or how challenging it is, or maybe learn something all over themselves. But with AI agent or vibe coding, it basically removes that part from your brain. So yes, it saves a lot of time for you. And yes, it creates more time for you to focus on more impactful and more value creation part. But the trade off here is also very obvious, especially if you're going with the approach with Apple, making the perfect first and then launch one goal. It is much better if you go with the launch first iterate later approach.
And by the way, Anthropic themselves, they're also doing that. And let's say the degree they're doing it is already reaching to the level, they don't even write a PRD now. They just do it right away and iterate on the fly and say, okay, what is the behavior after this change, conducted by the Claude Code, and then to change it from there onward. So they don't even write the PRD now. So yeah, big changes.
I guess we can relate a lot of that. But sorry, Juan, you go ahead. No, no, that's also a very cultural thing. I see like only pushing things that are perfect, versus like, yeah, we'll figure it out. Talking about the cultural part of it again, if tech is going to get automated to some extent, we still can automate difficult communication with our stakeholders. Now, you've lived in vastly different cultures. And one of the things that I found is very difficult, especially when you're dealing with more power distance is disagreeing productively. So do you have any tips for anyone listening in on how to deal with that? I
think the best way to do it is actually read the book, honestly, because I actually got this idea from the book, The Culture Map. It was quite popular and famous around three or four years ago in Berlin. And I strongly recommend it. Even if you say you don't have time to read it, at least find the infographic on Google image, or whatever search image you could find online, or maybe ask AI, ask you to explain the idea of power distance. And I have to say there's no one simple bullet to one size fit or distribution here. You have to adopt. And you need to factor in the cultural background of the first of all, the organization itself. And second of all, the audience you're going to express your disagreement with. And third, that's also the context there. So yes, context is definitely the key word here, no matter the cultural differences here, and also the context for AI. So context is the goal word here.
I suppose we always assume there's one culture, there's one country culture, but it gets very difficult, especially when you work in very international companies. How would you look at that from a leadership perspective? Like how do you make sure that everyone is heard, but there's still some sort of unity in how the team operates?
This is a million dollar question, man. First and foremost, regarding communications, I think the golden rule here is always keep it explicit, especially when it comes to writing, written communication. Because for whatever culture they have from a background, they have a very diverse combination for the country. They usually would develop into a very explicit communications culture and preference. For example, the US, it's actually a multi-ethnic country. There's a lot of different races they mix together in the States. That's why they also tend to be explicit communication culture. That being said, how do you make sure everyone is being heard? It is very tricky to manage. I would say if you want to do this perfectly, it will be a full-time job of itself. So for me, how I would operate there is I would establish our own team culture or our own organizational culture. Usually, the team culture would inherit a lot of property from the company culture. I
would try to build something on top of it and try to establish our own thing. Because this kind of team culture and the team ritual and team tradition, they are the artifact they hold us together as a group. We need to first establish a relationship and then we have this culture ongoing. Then, yes, we could establish good and frequent communication there. Of course, usually, as the leadership positions, I would always prioritize public communication, especially within team communication. Especially before making some important announcements, for example, scheduled maintenance or maybe there's an incident ongoing or outrageous to resolve, those I still need to get through with a team before I talk to the stakeholder.
Let alone talk to the big bosses. In addition to that, the regular one-on-one is also very vital. Those are actually the golden moments for building the very close distance of close counter way to build a relationship with your direct report. Sometimes, for your important stakeholders, especially executive level, that is also a good practice to establish some sort of a regular touch point. I actually learned that in my first year days, I started doing it with my direct report. Of course, a direct report, there will be like on a weekly basis. For important stakeholders, I try to do it on a monthly basis. Especially, there are certain things you want to talk to the executive of your stakeholder in a business with no filter. Then, we could synchronize what to tell the people who work in the front line and the individual contributor. Usually, this kind of one-on-ones with your stakeholder, they are unofficial. Of course, they should also be private
between you two. But I find this very, very useful. Last but not least, skip-level meetings with the manager of your manager, if you have one, is actually also very useful because you also got to have much bigger visibility of the whole organization, house the development of the business stone. You also got a good sense of direction of where the company is heading to and what does it mean for the rest of the company and what should we do in the data team if the business is having this involvement. That would be the combination of different things that need to be done together. But once again, this is more like my way of handling it. If you feel that you like the idea, definitely learn something from it. If not, you're also more than welcome to do your own thing. Because to me, for leadership and communication, they are more art than science.
It's like cooking. Like cooking, you can go super scientific. You could buy a temperature and say, "Okay, I need to cut this particular beef into like 5 centimeters per toothright. And I need to make sure when it's being fried, I need to be exactly at 70 degrees Celsius." Or you could go like a Brazilian or a lot of people. They just go and just wing it and keep cooking it. It feels like medium rare and then it's done. So it's more art than science.
You mentioned that you formed a culture with your team and you keep things explicit. Can you give us an example of how that works? Do you come up with a list of these are the commandments or do you just let people infer it by your actions?
Let me try to think of an example because I did a lot of things there. Let's take Retro, Retrospective as part of the ceremony of Agile. For Retrospective, usually what would be done, when it's done, that would be a Retrospective document there. This is actually one of the ways I have employed and invite everyone in the team to explicitly write down what they liked about the previous sprint and what they think it could be better in the previous sprint. Then everything got aggregated together and it would generate a document there. As time goes, I think we have around probably, because I've been there for four years already, probably some hundreds of documentation there. Those documentation would actually be written and make sure we don't repeat the same mistake in the past.
Also, those documentation would be referred whenever we are not sure what decision is being made. Usually, the discussion is more on, "We make the same mistakes over time." Usually. This is how it started. Once we start having this Retrospective exercise, leaving the documentation there, people start doing it more. Also, when I was in the first year, we also had this setup for two different teams in data, one team in Berlin and another team in Paris. It also means we would sometimes have to work asynchronously. By default, for example, if we write a message and communicate to each other, that would be in a written form as well. I mean, for us, it's very natural, very normal. It did tell us a bit more effort and time for the Paris team to get used to this accommodation. As time goes, we still managed to do it, but I have to say it wasn't easy because this is not natural for French working culture.
Interesting. I have you here and you're probably one of the people that I know with the most working with data management. Here's the thing. I think flying cars, right? I can give you the four principles of flying cars, but there's always a big gap between me more or less explaining how they are versus actually doing them. Can you tell us a little bit of, first of all, what do you think about data mesh? Does it work? Is it not more for you? Awesome. More importantly, how do you fill out those gaps between a book of 150 pages to actually running it?
How do I put it? Data mesh is, first of all, it needs your organization and data being scalable to a certain point to have this problem. You only see super huge companies like the Lister company, like Delivery Hero, HelloFresh, and actually, Zalando as well. They claim to be the first European company adopting Data Mesh. I don't know if it's true or not, to be honest, but I don't see it being challenged. Let's trust them. Anyway, it is a big company problem, especially if they also have a very high demand for data, especially for reporting and machine learning as well. That creates this problem to be solved by introducing Data Mesh. I have to say that Data Mesh by itself is a very good idea if you go through all the principles and the problem, especially in a German company, is always about execution.
You could talk through the principles with them very easily. When it comes to execution, it would require quite some creativity and flexibility and also maintaining the focus of the principle behind. I have to say, in the implementation, among these a couple of companies I've seen, some of them are very good, some of them they're absolute shit, and some of them they're in the middle. I would say Delivery Hero, they were still okay-ish when it comes to the implementation of Data Mesh. At least they have proper separation of production and staging environment. The only problem they had was actually monolithic repository. This is probably the busiest CI pipeline I have seen in the world. Every single minute they have 200 pending CI because they have a couple of hundred data people working on the same repository every single day. Just by running a simple check in the CI, it takes around two hours already in the trial CI I have seen. It's still
not that bad compared to HelloFresh. For HelloFresh, I think the worst part is the fragmentation of the toolings. Yes, by principle, they are doing what is meant by Data Mesh. For the data warehouse and the data platform, they stick to Impala, which has a very huge problem with scalability. Basically, every date around 4pm or 5pm in Berlin, I could forget about work. I'm running an inquiry because the SPOs are breaking up and they have a lot of ill-performing, curious running, and hitting Impala. We are sharing the same culture. Basically, we can forget about it. Another problem is that they also allow you to build a certain data mart on your own as an analysis, not as a data engineer.
What's the technology behind the materialized data marts from the analyst? They call it materialized view, which is very technically misleading. There's actually a table. This is a repository of their own Airflow. It actually runs via Spark, if I'm not mistaken. Basically, if you want to build your own model, you need to know the tool dialect in SQL. You need to know the Impala side of SQL. You also need to understand how Spark works. What happens to be in a very bad position back in the days is that my primary data source is actually the first implementation of Kafka. They ingest the Kafka events in Impala. And we make them with all the problems we just mentioned. It basically becomes impossible for me to conduct any meaningful analysis there. I also tried to run on a local machine by having my own database there. Sadly, back in the days, we did not have a DuckDB, which is my current favorite.
So yeah, it's impossible to run any data analysis there. It was a bad implementation. Also, another implementation I have just mentioned in the first year, it was a great new model for the ones. Personally, I think it is... I take proud of it pretty much actually, because I would say I still graph the idea of a data mesh and still adopt it for the unique of the company and make it work. So I would say even for data mesh, just understand it by principle, but don't take it literally word by word. It's meant to be a high level principle to inspire your thinking. And you are still the humans to make use of your own judgment, to make it land in your own context. Yeah, probably the
biggest problem with data mesh in my experience was that every type of company was trying to implement them regardless of their scale or the problems that they were having. I guess when you frame it like that, it makes all sense that for some organizations, it's much better fit than for others. Yeah, and
that's also one common people for data people actually, because what you just mentioned about the data mesh implementation is actually a problem being caused when us as data people, we think the technical thing before the business problem. Because the whole point of data mesh in the business sense is to ensure faster delivery of the data product and better quality of the delivery of the data product. Because whoever is building them and maintaining them, they should be the domain expert. For example, operation, they should have their own data product of their own curated data mart, or maybe their own dashboard, or maybe their own algorithm. And whatever they serve, they should be instantly reusable by marketing, for example.
That being said, there are some nuance that we always need to remember. The point is to materialize the expertise in operation in data and let another domain who does not have the expertise in ops, for example, marketing, to make use of this data product to speedily do whatever they want to do. That was the idea. And I don't think a lot of people actually get this idea. audio]
So, Jimmy, what's next for you? And where can people find you?
What's next for me? Well, as you mentioned at the beginning of this session, I'm in the progress of building Data Biz. And that would be my AI and data consultancy. That's actually the first entry point for me. Because I've seen a lot of consultancy, like dltHub and Fishtown Analytics, which everyone knows them as dbt Labs and also Metabase and the other companies, they all start out as a consultancy and they build their own data product, open source data product for the rest of the world to consume and to enjoy it. So I plan to do the same, but of course, it's got a tick time. And where can you find me? Of course, you can find me on LinkedIn and I'm still physically based in Berlin, Germany as well. So make sure you say hi and reach out via LinkedIn and we could schedule a time to meet in person.
Nice. All right. Well, thank you for coming over, Jimmy. We had a lovely conversation. We'll make sure to get all your contacts and details in the description or down below, wherever people are watching this. And yeah, very excited to see your next steps.