Probably actually reading/writing

March 5, 2020 — January 15, 2025

Suspiciously similar content

Figure 1

Stuff that I am currently actively reading or otherwise working on. If you are looking at this, and you aren’t me, you may need to consider re-evaluating your hobbies.

1 Refactoring

I need to reclassify the bio computing links; that section has become confusing and there are too many nice ideas there not clearly distinguished.

2 Currently writing

Not all published yet.

  1. So you’ve just joined a union

  2. When is computation statistical, in the sense that we know the dynamics of a population of solutions, even when we cannot do the computations? Not quite sure of the scope of this, so let’s use some examples to flesh it out:

    1. Trading equities. We cannot know what trades everyone is making, but we can do a good job of pricing options under no-arbitrage assumptions and the like, even though some of the calculations made by people pricing things in the market will be very complicated, and in fact far more complicated than ours. The no-arbitrage assumptions are not strictly true, but the returns to complexity in finding arbitrage opportunities seem to diminish in compute, or something like that, and in the wash it’s pretty similar.
    2. Scaling laws: we cannot know what exact computations an LLM will do, but we can estimate how well it will do them remarkably well under a certain data-parameter-train-compute budget.
    3. Algorithmic statistics and pseudorandomness are about the statistical behaviours of certain classes of algorithms, in a broader sense, wherein they become near-indistinguishable from randomness, in certain technical senses
    4. …

  3. AI Safety

    1. metrics that come apart from their goals
    2. domestication of humans
    3. Causal agency.

  4. Foundation models and their world models

  5. Community building

    1. Collective care
    2. Social calendaring
    3. Psychological resilience

  6. Reality gap

  7. Continual learning.

  8. Is academic literary studies actually distinct from the security discipline of studying side-channel attacks?

  9. Goodhart coordination

  10. Structural problems are hard let’s do training programs

  11. Extraversion

  12. Is residual prediction different from adversarial prediction?

  13. Science communication for ML

  14. Human superorganisms

    1. Moral orbits.
    2. Revisit probability collectives
    3. Movement design
    4. Returns on hierarchy
    5. Effective collectivism
    6. Alignment
    7. Emancipating my tribe, the cruelty of collectivism (and why I love it anyway)
    8. Institutions for angels
    9. Institutional alignment
    10. Beliefs and rituals of tribes
    11. Where to deploy taboo
    12. The Great Society will never feel great, merely be better than the alternatives
    13. Egregores etc
    14. Player versus game
    15. Something about the fungibility of hipness and cash
    16. Monastic traditions

  15. Approximate conditioning

  16. Nested sampling

  17. What even are GFlownets?

  18. Public sphere business models

  19. How to do house stuff (renovation etc)

  20. Power and inscrutability

  21. Strategic ignorance

  22. What is an energy-based model?? tl;dr branding for models that handle likelihoods through a potential function which is not normalised to be a density. I do not think there is anything new about that per se?

  23. Funny-shaped learning

    1. Causal attention
    2. Graphical ML
    3. Gradient message passing
    4. All inference is already variational inference

  24. Human learner series

    1. Which self?

    2. Is language symbolic?

    3. Our moral wetware

    4. Is is ought

    5. Morality under uncertainty and computational constraint

    6. Superstimuli

    7. Clickbait bandits

    8. Correlation construction

    9. Moral explainability

      1. Burkean conservatism is about unpacking when moral training data is out-of-distribution.
      2. Something about universal grammar and its learnable local approximations, versus universal ethics and its learnable local approximations. Morality by template, computational difficulty of moral identification. Leading by example of necessity.

    10. Righting and wronging

    11. Akrasia in stochastic processes: What time-integrated happiness should we optimise?

    12. Comfort traps ✅ Good enough for now

    13. Myths ✅ a few notes is enough

  25. Classification and society series

    1. Constructivist rationalism
    2. Affirming the consequent and evaporative tribalism
    3. Classifications are not very informative
    4. Adversarial categorization
    5. AUC and collateral damage
    6. Bias and base rates
    7. Decision theory
    8. Decision theory and prejudice

  26. Shouting at each other on the internet series (Teleological liberalism)

    1. Modern politics seems to be excellent at reducing the vast spectrum of policy space to two mediocre choices then arguing about which one is worse. What is this tendency called?
    2. The Activist and decoupling games, and game-changing
    3. On being a good weak learner
    4. Lived evidence deductions and/or ad hominem for discussing genetic arguments.
    5. Diffusion of responsibility — is this distinct from messenger shooting?
    6. Iterative game theory of communication styles
    7. Invasive arguments
    8. Coalition games
    9. All We Need Is Hate
    10. Speech standards
    11. Startup justice warriors/move fast and cancel things
    12. Pluralism ✅

  27. Learning in context

    1. Interaction effects are what we want
    2. Interpolation is what we want
    3. Optimal conditioning is what we want
    4. Correlation construction is easier than causation learning

  28. Epistemic community design

    1. Scientific community
    2. Messenger shooting
    3. On being a good weak learner
    4. Experimental ethics and surveillance
    5. Steps to an ecology of mind
    6. Epistemic bottlenecks is probably in this series too.
    7. Ensemble strategies at the population level. I don’t need to guess right, we need a society in which people in aggregate guess in a calibrated way.

  29. Epistemic bottlenecks and bandwidth problems

    1. Information versus learning as a fundamental question of ML. When do we store exemplars on disk? When do we gradient updates? How much compute to spend on compressing?
    2. What is special about science? One thing is transmissibility. Can chatGPT do transmission? Or is it 100% tacit? How does explainability relate to transmissibility?

  30. DIY and the feast of fools

  31. Tail risks and epistemic uncertainty

    1. Black swan farming
    2. Wicked tail risks
    3. Planning under uncertainty

  32. Economic dematerialization via

    1. Enclosing the intellectual commons
    2. Creative economy jobs

  33. Academic publications as Veblen goods

  34. Stein variational gradient descent

  35. Edge of chaos, history of

  36. X is Yer than Z

  37. But what can I do?

    1. Starfish problems
    2. Ethical consumption
    3. Prefigurative politics

  38. Haunting and exchangeability. Connection to interpolation, and individuation, and to legibility, and nonparametrics.

  39. Doing complicated things naively

  40. Conspiracies as simulations

  41. Something about the limits of legible fairness versus metis in common property regimes

  42. The uncanny ally

  43. Elliptical belief propagation

  44. Strategic ignorance

  45. Privilege accountancy

  46. Anthropic principles ✅ Good enough

  47. You can’t talk about us without us ❌ what did I even mean? something about mottes and baileys?

  48. Subculture dynamics ✅ Good enough

  49. Opinion dynamics (memetics for beginners) ✅ Good enough

  50. Table stakes versus tokenism ✅

  51. Iterative game theory under bounded rationality ❌ too general

  52. Memetics ❌ (too big, will never finish)

  53. Cradlesnatch calculator ✅ Good enough

3 music stuff

4 Misc

5 Workflow optimization

6 graphical models

7 “transfer” learning

8 Custom diffusion

9 Commoncog

10 Music skills

11 Internal

12 ICML 2023 workshop

13 Neurips 2022 follow-ups

  1. Arya et al. (2022) — stochastic gradients are more general than deterministic ones because they are defined on discrete vars
  2. Rudner et al. (2022)
  3. Phillips et al. (2022) — diffusions in the spectral domain allow us to handle continuous function valued inputs
  4. Gahungu et al. (2022)
  5. Wu, Maruyama, and Leskovec (2022) LE-PDE is a learnable low-rank approximation method
  6. Holl, Koltun, and Thuerey (2022) — Physics loss via forward simulations, without the need for sensitivity.
  7. Neural density estimation
  8. Metrics for inverse design and inverse inference problems - the former is in fact easier. Or is it? Can we simply attain forward prediction loss?
  9. Noise injection in emulator learning (see refs in Su et al. (2022))

14 Conf, publication venues

15 Neurips 2022

16 Neurips 2021

17 Music

Nestup / cutelabnyc/nested-tuplets: Fancy javascript for manipulating nested tuplets.

18 Hot topics

19 Stein stuff

20 newsletter migration

21 GP research

21.1 Invenia’s GP expansion ideas

22 SDEs, optimization and gradient flows

Nguyen and Malinsky (2020)

Statistical Inference via Convex Optimization.

Conjugate functions illustrated.

Francis Bach on the use of geometric sums and a different take by Julyan Arbel.

Tutorial to approximating differentiable control problems. An extension of this is universal differential equations.

23 Career tips and metalearning

24 Ensembles and particle methods

25 Foundations of ML

So much Michael Betancourt.

26 nonparametrics

27 References

Arya, Schauer, Schäfer, et al. 2022. “Automatic Differentiation of Programs with Discrete Randomness.” In.
Gahungu, Lanyon, Álvarez, et al. 2022. “Adjoint-Aided Inference of Gaussian Process Driven Differential Equations.” In.
Holl, Koltun, and Thuerey. 2022. “Scale-Invariant Learning by Physics Inversion.” In.
Lai, Takida, Murata, et al. 2022. “Regularizing Score-Based Models with Score Fokker-Planck Equations.” In.
Nguyen, and Malinsky. 2020. “Exploration and Implementation of Neural Ordinary Differential Equations.”
Phillips, Seror, Hutchinson, et al. 2022. “Spectral Diffusion Processes.” In.
Rudner, Chen, Teh, et al. 2022. “Tractable Function-Space Variational Inference in Bayesian Neural Networks.” In.
Su, Kempe, Fielding, et al. 2022. “Adversarial Noise Injection for Learned Turbulence Simulations.” In.
Wu, Maruyama, and Leskovec. 2022. “Learning to Accelerate Partial Differential Equations via Latent Global Evolution.”