Back to resources

Creating From Scratch Without Any Prior Knowledge

Managing Expectations
Collaboration

10 September, 2021

Christophe Blefari

Christophe Blefari

Data Coach at Self Employed

Christophe Blefari, Data Coach, describes how to approach creating something new when one doesn’t know much about the domain, technology, or what their next step should be.

Problem

I have been in this situation a great many times. I had to create a data lake using AWS or Google cloud platform. I got data from production and put it in a data warehouse, let’s say Snowflake or Ratchet. I would have to create a data streaming platform using Kafka and put some machine learning algorithms in production too. Then I had to develop the platform, train models, and publish predictions.

It all looks seamless, right? But where to start from when you are not familiar with the domain or technology?

Actions taken

I would reach out for help wherever possible. To start with, I would reach out to my peers -- either those within the company or outside. I wouldn’t necessarily stick to my narrow field of expertise only; great ideas and inspiration could come from anywhere. I would also not shy away from approaching the competition because, in today’s world, people who were competitors a year ago may end up working on the same projects in no time. I most often approach peers outside my company through LinkedIn. Also, looking for help through mentorship platforms such as Plato should also be an opportunity to be explored.

If one is using AWS or GCP, they can reach out directly to the company and ask them to help them out. Sometimes a free trial could be obtained, or one will be directed to a consultancy firm that implements their solutions and can help. For example, Google paid ten days of consultancy to the firm they were working with to help us onboard. For ten days, we could toy around with a new tool free of charge.

Further on, I would read whatever I could get my hands on: documentation, live content on YouTube, blogs from reputable tech companies, etc. Taken all together, this should be enough for one to get a basic understanding of the domain and start proliferating some ideas of their own. However, this is the most difficult moment. A person would acquire all of that great knowledge and then will have to take a leap and start creating something on their own. What makes this leap difficult is that in this phase, one needs to make some decisions. After studying different approaches, a person could be overwhelmed with multiple ideas, not knowing which one to choose. To choose the right one, I created a matrix with a number of variables: cost, time to implement, internal expertise, scalability, availability of skilled engineers, etc.

Finally, whenever I build something from scratch, I like to take a pragmatic approach. First, I do an MVP of the platform, after which I start building things incrementally. I find it useful to break down one large project into several smaller segments, each being contextualized. For example, if I need to do a large data platform, I would create it first for the accounting team, then the marketing team, or even one or only several use cases for the marketing team. If it works for one use case, I will keep adding other use cases. Only once I am certain of what I am building, I would start thinking about a roadmap.

Lessons learned

  • There is no way for you to try everything. In the best-case scenario, you should go with three approaches at most. In today’s engineering world, there are too many solutions for the same problem. People often get stuck in a try-mode and end up indefinitely trying out different things. You need to limit a period over which you will be trying new things, and once it is over, you will make the decision.
  • Be pragmatic. For me, being pragmatic means start small and iterate. See for yourself if your solution works on a small scale and if so, scale it up.
  • Read the documentation. We are using more and more tools, and all of those tools are coupled with documentation for a reason. Each time we have a problem, we go and Google it. If you read the documentation first, you will save yourself a lot of time. Read the documentation before you try to do the implementation. Act the same way as when you are assembling furniture -- you read a manual before you start screwing bolts.
  • People are in general nice and want to help. So, don’t shy away from asking for help.

Discover Plato

Scale your coaching effort for your engineering and product teams
Develop yourself to become a stronger engineering / product leader


Related stories

Myth Busting

10 December

Supporting principles on why being data led (not driven) helps with the story telling.

Alignment
Managing Expectations
Building A Team
Leadership
Collaboration
Productivity
Feedback
Psychological Safety
Stakeholders
Vikash Chhaganlal

Vikash Chhaganlal

Head of Engineering at Xero

The Growth Mindset in Modern Product Engineering

28 November

The impact you can have with a Growth Mindset' and the factors involved in driving orchestrated change.

Building A Team
Leadership
Collaboration
Feedback
Ownership
Stakeholders
Vikash Chhaganlal

Vikash Chhaganlal

Head of Engineering at Xero

How to improve engagement and retention in remote engineering teams?

25 October

Mrunal Kapade, an Engineering leader, based in Silicon Valley, shares tips that helped reduce attrition in the remote engineering teams while leading multiple teams from startups to Fortune 500 companies.

Remote
Company Culture
Collaboration
Motivation
Team Processes
Mrunal Kapade

Mrunal Kapade

Director of Engineering at Inspire Energy

High Performance Team in Action

13 October

A high performance team refers to “ a group of goal-focused individuals with specialized expertise and complementary skills who collaborate, innovate and produce consistently superior results.”

Managing Expectations
Building A Team
Company Culture
Feedback
Coaching / Training / Mentorship
Praveen Cheruvu

Praveen Cheruvu

Senior Software Engineering Manager at Anaplan

Scaling a Team in Two Parts: The Product and Manager

2 August

Viswa Mani Kiran Peddinti, Sr Engineering Manager at Instacart, walks through his experience scaling a team, product and his skills as a leader.

Managing Expectations
Product
Scaling Team
Leadership
Meetings
Viswa Mani Kiran Peddinti

Viswa Mani Kiran Peddinti

Sr Engineering Manager at Instacart