Update README to align with current scripts/process

[0xdevalias]

I haven't really updated the 'Helper Scripts' / 'Getting Started' sections of the README for quite a while, and so they aren't really fully aligned to how I am actually doing things these days (more based on the older manual'ish methods, or maybe the first iteration of semi-automation)

It would be good to figure out what is completely outdated, what is still relevant but the 'old way' of doing things, and what is the 'new/current way' of doing things; and then update the README to capture that knowledge rather than it being locked up in my head/similar.

---

At a very high level, off the top of my head, my current process is basically:

• Load ChatGPT and let my userscript check/notify me if there are any new script files
  • https://github.com/0xdevalias/userscripts/tree/main/userscripts/chatgpt-web-app-script-update-notifier
• If there are new scripts, use the 'Copy ChatGPT Script data to clipboard' menu option on Tampermonkey
• Run the following script to get a filtered list of the JSON (with dates) and list of URLs to be downloaded:

  • pbpaste | ./scripts/filter-urls-not-in-changelog.js --json-with-urls

  • # Example
    ⇒ pbpaste | ./scripts/filter-urls-not-in-changelog.js --json-with-urls
    {
      url: 'https://cdn.oaistatic.com/_next/static/chunks/pages/_app-783c9d3d0c38be69.js',
      date: '2024-02-24T02:18:13.376Z'
    }
    {
      url: 'https://cdn.oaistatic.com/_next/static/chunks/webpack-2e4c364289bb4774.js',
      date: '2024-02-24T02:18:13.376Z'
    }
    {
      url: 'https://cdn.oaistatic.com/_next/static/WRJHgIqMF1lNwSuszzsvl/_buildManifest.js',
      date: '2024-02-24T02:18:13.376Z'
    }
    {
      url: 'https://cdn.oaistatic.com/_next/static/WRJHgIqMF1lNwSuszzsvl/_ssgManifest.js',
      date: '2024-02-24T02:18:13.376Z'
    }
    https://cdn.oaistatic.com/_next/static/chunks/pages/_app-783c9d3d0c38be69.js
    https://cdn.oaistatic.com/_next/static/chunks/webpack-2e4c364289bb4774.js
    https://cdn.oaistatic.com/_next/static/WRJHgIqMF1lNwSuszzsvl/_buildManifest.js
    https://cdn.oaistatic.com/_next/static/WRJHgIqMF1lNwSuszzsvl/_ssgManifest.js

• Copy the output of this command and paste it into SublimeText as a scratch pad/reference
• Copy the list of URLs, then run the following command

  • pbpaste | ./scripts/add-new-build-v2.sh 2>&1 | subl

  • This will does a bulk of the automation of checking/downloading the URLs, extracting additional URLs from the _buildManifest.js and webpack.js + downloading those, unpacking/formatting the downloaded files, generating a copy/pasteable CHANGELOG entry, etc.
  • Note that as part of running this script, it will ask for the date of the build (from the above JSON) to be input at one point before the CHANGELOG entry is generated
• Manually copy/paste the generated CHANGELOG entry into the CHANGELOG.md file, generate and add the updated link in the Table of Contents, then modify the entry to add manual analysis notes, etc
  • https://derlin.github.io/bitdowntoc/
• Commit/push the downloaded files + updated CHANGELOG
• Potentially write a tweet about the update linking back to the CHANGELOG update, etc
  • If we do, then we should also edit the CHANGELOG again to add a link to that Tweet/thread.
  • Sometimes I will also make a crossposted update on Reddit / LinkedIn / HackerNews / etc; if I do, I tend to also link to those posts in the Tweet thread (and maybe sometimes in the CHANGELOG as well, but I don't think I have bothered with that much lately)

There might be bits in that that aren't perfectly documented; or little snippets of nuance that i've missed, but roughly that is my process currently.

---

In manually reviewing the diffs to add my 'manual analysis' to the CHANGELOG, there is often a lot of 'diff churn' noise from the minified variable names changing in the webpack build/etc. I've been working on some new scripts that help minimise that; which I haven't currently pushed, but you can see some of my notes about them in this issue:

• Explore AST based diff tools #3

Currently, I sort of roughly/hackily run them with a command similar to this:

diffmin-wc-raw () { git diff --diff-algorithm=patience $1 | wc -l; };

diffmin-wc () { git diff --diff-algorithm=patience $1 | ./scripts/ast-poc/diff-minimiser.js 2>/dev/null | wc -l; };

diffmin () { git diff --diff-algorithm=patience $1 | ./scripts/ast-poc/diff-minimiser.js 2>/dev/null | delta; };

# diffmin-wc-raw unpacked/_next/static/chunks/pages/_app.js
# diffmin-wc unpacked/_next/static/chunks/pages/_app.js
diffmin unpacked/_next/static/chunks/pages/_app.js

See Also

• Automate checking for new builds with GitHub action/similar #8