Skip to content

Commit

Permalink
📝
Browse files Browse the repository at this point in the history
  • Loading branch information
davidgasquez committed Dec 14, 2022
1 parent 12007fd commit db225ad
Show file tree
Hide file tree
Showing 2 changed files with 6 additions and 2 deletions.
5 changes: 5 additions & 0 deletions Data/Data Engineering.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,11 @@

- Gather as much domain knowledge as possible. Technical knowledge is not enough.
- If it can be solved with SQL, stick to SQL.
- To be successful, data engineers need to care about:
- **Fidelity:** how reliably can data be transferred and stored without corruption or loss?
- **Capacity:** how much data can be moved and how quickly?
- **Reliability:** how well can systems recover from outages and incidents?
- **Speed of execution:** how quickly can you get a new data source up and running?

## Data Pipelines

Expand Down
3 changes: 1 addition & 2 deletions Teamwork.md
Original file line number Diff line number Diff line change
Expand Up @@ -52,8 +52,7 @@
2) Remove more than you add.
3) Optimize what works.
4) Shorten iteration cycles. **[Boyd's Law of Iteration](https://blog.codinghorror.com/boyds-law-of-iteration/): speed of iteration beats quality of iteration**.
5) [[Automation | Automate]].
6) Standards, a straight pipe is easier to fix than a curved one
5) [[Automation | Automate]] and keep standards.
- Keep great global [[coordination]] and incentivize local experimentation.
- Being able to run small and compounding experiments (on the product or company [[processes]] and systems) is important.
- [Some experiments won't work](https://www.lesswrong.com/posts/97LgacucCxmyjYiNT/the-archipelago-model-of-community-standards). But oftentimes it _feels_ like it wont work when in fact you just haven't stuck with it long enough for it to bear fruit. This is hard enough for _solo_ experiments. For group experiments, where not just one but _many_ people must all try a thing at once and _get good at it_, all it takes is a little defection to spiral into a mass exodus.
Expand Down

0 comments on commit db225ad

Please sign in to comment.