From 1f7f14d312e4052e9e84176ad4e914102761f8eb Mon Sep 17 00:00:00 2001 From: David Gasquez Date: Sat, 11 Jan 2025 11:29:53 +0100 Subject: [PATCH] =?UTF-8?q?docs:=20=F0=9F=94=A7=20elaborate=20forecast=20v?= =?UTF-8?q?alidation=20and=20add=20model=20market=20idea?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Adds more detail about prediction validation (target vs predicted errors), fixes typos (society), and proposes a competing AI model market with rewards based on prediction accuracy. --- Datathons.md | 2 +- Future.md | 3 ++- Time.md | 2 +- 3 files changed, 4 insertions(+), 3 deletions(-) diff --git a/Datathons.md b/Datathons.md index 7343ca5..29baa97 100644 --- a/Datathons.md +++ b/Datathons.md @@ -10,7 +10,7 @@ 6. Plot learning curves ([sklearn](https://scikit-learn.org/stable/modules/learning_curve.html) or [external tools](https://github.com/reiinakano/scikit-plot)) to avoid overfitting. 7. Plot real and predicted target distribution to see how well your model understand the underlying distribution. Apply any postprocessing that might fix small things. 8. Tune hyper-parameters once you've settled on an specific approach ([hyperopt](target distribution), [optuna](https://optuna.readthedocs.io/)). -9. Plot and visualize the predictions (histograms, random prediction, ...) to make sure they're doing as expected. Explain the predictions with [SHAP](https://github.com/slundberg/shap). +9. Plot and visualize the predictions (target vs predicted errors, histograms, random prediction, ...) to make sure they're doing as expected. Explain the predictions with [SHAP](https://github.com/slundberg/shap). 10. Think about what postprocessing heuristics can be done to improve or correct predictions. 11. [Stack](https://scikit-learn.org/stable/auto_examples/ensemble/plot_stack_predictors.html) classifiers ([example](https://www.kaggle.com/couyang/featuretools-sklearn-pipeline#ML-Pipeline)). 12. Try AutoML models. For tabular data: [TPOT](https://github.com/EpistasisLab/tpot), [AutoSklearn](https://github.com/automl/auto-sklearn), [AutoGluon](https://auto.gluon.ai/stable/index.html), Google AI Platform, [PyCaret](https://github.com/pycaret/pycaret), [Fast.ai](https://docs.fast.ai/), [Alex](https://github.com/Alex-Lekov/AutoML_Alex).For time series: [AtsPy](https://github.com/firmai/atspy), [DeepAR](https://docs.aws.amazon.com/forecast/latest/dg/aws-forecast-recipe-deeparplus.html), [Nixtla's NBEATS](https://nixtlaverse.nixtla.io/neuralforecast/models.nbeats.html), [AutoTS](https://github.com/winedarksea/AutoTS). diff --git a/Future.md b/Future.md index 2e0b85f..edaeea1 100644 --- a/Future.md +++ b/Future.md @@ -9,7 +9,7 @@ History teaches us that in 100 years from now [[Openness|some of the assumptions - Give birth without advanced assistance. - Not caring for all the [animal suffering in the wild](https://longtermrisk.org/the-importance-of-wild-animal-suffering/). - Nature is not safe! The default is suffering. The current mentality is that nature is good and disruptions from nature are bad. -- The ignorance of Social Media and its [full impact on s_o_ciety](https://twitter.com/M_B_Petersen/status/1483457679800651787). +- The ignorance of Social Media and its [full impact on society](https://twitter.com/M_B_Petersen/status/1483457679800651787). - Is "being bad for society" an emergent property of social networks as they grow? - Current Voting Systems. - Not relying more into tools like Prediction Markets (e.g: [to spot papers that might not replicate](https://vitalik.eth.limo/general/2024/11/09/infofinance.html)). @@ -41,3 +41,4 @@ History teaches us that in 100 years from now [[Openness|some of the assumptions - Prolly/Merkle Trees - Differential/Timely Dataflow - Zero-Knowledge Proofs +- Have a market of ML/AI models each competing to predict an outcome. When the outcome actually happens, rank them on a leaderboard by how close their predictions were and reward accordingly. diff --git a/Time.md b/Time.md index 3634adb..1e25e98 100644 --- a/Time.md +++ b/Time.md @@ -16,7 +16,7 @@ Time is the most valuable and least replaceable resource. Just like money, [time - Do the tasks in the right order! One strategy might be starting with the one that could make the others irrelevant or easier. - Time-boxing is [[Planning]] how you spend your days in advance and it's so effective because it allows you to iterate. If you didn't complete everything you outlined, you know exactly why -- because you've documented how you planned to spend your time. - You'll never have any more time. You have, and have always had, all the time there is. [How you spend your time is a choice](https://leebyron.com/4000/). -- Success can be measured in the percentage of time you have under your control. [To achieve success](https://blog.samaltman.com/how-to-be-successful): +- Success can be measured in the percentage of time you have under your control: - Compound yourself. Compounding is magic. Keep long-term thinking with a broad view of how different systems in the world are going to come together. - Learn to think independently. - Make it easy to take risks. Most people overestimate risk and underestimate reward. It's often easier to take risks early in your career; you don't have much to lose, and you potentially have a lot to gain.