Nathan Braun

nathanbraun.com - Tech Tools

Thu, 16 Jan 2025 00:00:00 GMT

Writeup of Tech Tools

Notes for HS Talk

Tue, 16 May 2023 00:00:00 GMT

Notes for HS Talk

The problem with advice

depends on the person, their goals and situation
really good advice that applies to almost everyone (e.g. don't try heroin) is mostly well known

So I can talk about my experience, what's worked for me and what I wish I'd done differently or known when I was in HS, but your mileage may vary.

Will start with quick overview on my background then we can branch out to whatever you guys think would be most helpful.

My experience

Neenah HS, 2005; UW Madison 2009, Montana State 2011
environmental economics/BP Oil Spill
learning to code/side projects
data science day jobs
working for myself teaching programming and data science with sports

What would be useful?

Happy to expand on any of these, or anything else.

What do programmers mostly do?
How did I teach myself to code?
What programming language should you learn?
What's the connection between Java/programming in HS and businesses/what programmers do day to day?
- e.g. how are lists, bools, arrays, objects useful?
What's a CS degree like?
What is data science?
What kind of math do you use in programming?
What I like about programming:
- it's fun; like working on puzzles
- it let's you build things -- app, website, analysis -- which is satisfying
- it's everywhere
- might be the field that cares the least about formal credentials
- very powerful skillset
What I like about entrepreneurship:
- very minimal BS and wasted time
- in theory, doesn't have to be a connection between time and results
- easy to see how you're doing
- requires a variety of skills
General life advice
Mistakes and misconceptions
Tools I'm happy I learned/wish I learned earlier
- Vim - type faster
- Anki - memorize anything

Resources

Essays and articles

general life advice essays:
- Paul Graham (famous programmer, investor, billionaire) - notes for a HS talk
- Laura Deming - advice for ambitious teens
- Ben Kuhn has:
  - advice for people that just like him
  - a collection of his favorite life advice essays
- Patrick Collison (billionaire, founder of Stripe) - advice for 10-20 year olds
- Alex Guzey has some general advice too
programming specific essays:
- Patrick Mackenzie is a programmer/author who has a lot of good stuff, including:
  - career advice for programmers (recommended!)
  - tips for emailing busy people
- Always be Coding is an inspirational/good essay for people really into coding
- You don't need to work on hard problems by Ben Kuhn
- Paul Graham has a bunch on programming

Books

general
- Principles - Ray Dalio
- Deep Work - Cal Newport
- Superhuman by Habit - Tynan
- How to Fail at Everything and Still Win Big - Scott Adams
entrepreneurship:
- Start Small, Stay Small - Rob Walling (new book coming out too)
coding:
- Structure and Interpretation of Computer Programmers - famous/free coding book, also available in javascript version
- Crafting Interpeters - I haven't read this but it gets good reviews and if I was planning on majoring in CS I would (might do it anyway)

]]>

Money as a Database for Resource Allocation

Wed, 15 Feb 2023 00:00:00 GMT

"Money is a database for resource allocation"

I think I have a pretty deep understanding of what money actually is on a day to day basis because of Paypal...
...think of money as information. People often will think of money as having power in and of itself. It does not.
Just think of money as a database for resource allocation across time and space.
And then in what form should that database or data system — what form would be most effective? You're operating an economy and you need to have something that allows for efficient value ratios between products and services. You have this massive number of products and services and you can't just barter, that would be extremely unwieldy. You need something that gives you a ratio of exchange between goods and services and then something that allows you to shift obligations across time."

This is from an interview Elon Musk did with Lex Fridman in 2021. Elon has said something like this a few times. Predictably, he gets crap for it.

"Billionaire Elon Musk calls money 'Just an entry in a database' as fortune continues to soar." - The Independent

"Only the world's richest man can say money has no power!" - Twitter

What this means

When Elon says, "money just an entry in a database" he's not saying it in the way people say "age is just a number" or that if he logs into his Chase checking account and sees a $142B balance that that number is in some database on Chase's servers.

Instead, what I think means is that anytime you buy or sell something (or steal, or give), what you're really trading is resources. Money (technically prices) is information about these resources relative worth.

So as I write this, the idea of money as information allows me to say one bushel of wheat ($8.61) is worth 2.7 gallons of gas (at $3.15 per gallon), or 29 minutes of an Amazon warehouse employees time (at $17.84 per hour).

These are the ratios of exchange between goods and services that Elon is talking about.

Time and Space

Like Musk says, the money db is necessary to move value across time and space.

If I grow some tomatoes, those are ripe, juicy resources, ready to consume now. They're not going to last forever. But I can "sell" them to you, i.e. trade for money.

How does the money paid for tomatoes relate to value?

One answer is the value of money >= tomatoes for me (since I sold them) and money <= tomatoes for you (since you bought them).

But again, because money doesn't have any value in and of itself, we're really comparing the resources we can buy with $5 to to the tomatoes.

Moving value across space is a pretty easy extension of value across time. Unless I sell my tomatoes then literally stand in one spot (maybe waiting for a food truck drive by or something?), any shift in value across time is also a shift across space.

Is it really a database?

Although I think it's catchy, calling it a "database" is a bit misleading. It's not a database in the sense that it's persistent. If you pay me $5 cash for a carton of tomatoes there's no record of that anywhere. Musk's alternative framing, that "money is information" is probably more accurate.

Also, the information we get is one particular facet, namely the marginal value of the resource. Marginal basically means "the next". Right now I have a house, with running water + a bunch of seltzers in the fridge. That's all very relevant for how I value a marginal bottle of water I might buy from the store (not that highly).

Marginal value (or cost) exists on the producer side too. Each bottle of water costs something to make. If I need to build a giant water purifying plant, maybe the first one costs a lot, then they get cheaper after that. Maybe eventually (depending on how many I'm selling and where "the next" is) they get more expensive as water becomes harder to find.

The point is, a price as an entry in the money DB is at a very specific and context dependent point (the margin). It doesn't tell us e.g. the total value of all the water on earth and let us compare that to the total value of diamonds.

If you think about it, this makes sense. The value of something (even in terms of other resources) very much depends on how many you already have + what else is going on.

The software behind the money database

Musk thinks of different monetary systems as sort of the "software" this database runs on. The current, US government as money supply system (in practice a lot banks/credit card companies running "COBOL in batch mode") works pretty well.

Historically, one problem with government as money database is that, as Musk says, "governments [dilute] the money supply as a pernicious form of taxation." I.e. governments have political incentives to print money. That's because people like government spending a lot more than they like paying taxes, and why some people advocate e.g. a gold standard.

Cryptocurrency like Bitcoin could be another alternative. But it has drawbacks too, probably ones more serious than the risk of government dilution. As Musk says:

There is a fundamental issue with Bitcoin in its current form in that it's transaction volume is very limited. The latency for a properly confirmed transaction is too long. Much longer than you'd like. So it's not great for a transaction volume standpoint or a latency standpoint. So it is perhaps useful to solve an aspect of the money database problem, which is a store of wealth or an accounting of relative obligations, but it is not useful as a currency.

]]>

A Simple Explanation of Regression to the Mean

Fri, 09 Dec 2022 00:00:00 GMT

A Simple Explanation of Regression to the Mean

One concept I didn't really "get" for a long time was regression to the mean. I think I have some intuition for it now though and — in the spirit of my other simple explanations thought I'd write it up here.

Explanation

Say you have a fair coin, and you flip it 10 times and count the number of heads.

The number of heads in that 10 flips follows a binomial distribution which looks something like this:

There's about a 4% you get 2 (or 8) heads, 12% you get 3 (7), etc.

Now get 1000 different people to do that same set of 10 flips with their own unbiased coins. Just because of how probabilities work, about 40 of the people will flip two heads, about 120 three, etc.

Regression to the mean just means that — if these "extreme" 2 heads people continue flipping their coins, the low probability of heads is unlikely to continue.

If you flip a coin 10 times — regardless of what happened before — there's a 95% chance you'll get more than 2 heads, and a 90% chance you get between 3-7 heads.

So if these forty 2-head flippers all flip another 10 coins, 90% of them will flip something closer to the mean.

That's the key I think, recognizing "extreme" events as low probability events to begin with, and realizing that — if they really are low probability events and you continue drawing from them — you're likely to see results closer to the mean.

This example is easier because we have a coin we know is fair and a bunch of events where we know some are bound to be extreme. In real life, with smaller samples and unknown distributions it's harder because you don't necessarily know if you got an extreme draw from your distribution, or whether the underlying probability is different from what you thought.

For example: in 2016 the US elected Donald Trump, who was an "extreme" candidate (probability-wise). Was that a fluke draw and should we expect a regression to the mean where the US (or GOP) selects a more traditional candidate next time? Or was it some underlying shift where the probabilities changed and it's likely we see more of the same in the future? Hard to say.

]]>

Percentile Feedback

Thu, 08 Dec 2022 00:00:00 GMT

Percentile Feedback

A percentile feedback graph is a productivity tool for increasing your motivation. It shows you the portion of the day you've worked so far and how that compares to previous days.

"It's 11 AM and you've worked 2.5 hours; that's more than you've worked (as of 11 AM) on 75% of previous days!"

The bottom line is that you get a neat line that goes up when you work, and starts going down when you don't. The nice thing is you can always improve it — even if you've had a rough morning and line is starting low, things are redeemable, just work and it'll go up.

(Note: it's a line because each percentile — 75% — is calculated at some time — 11 AM. When you connect these series of points through the day it shows a line.)

The idea is you have it up so you can see it while you're working. It looks like this:

So in this example (which comes from Alex Vermeer), it's almost 7 pm (x axis) and Alex has worked about 40% of the day so far (y axis). Based on logs of previous days and times worked, this is better than 57% of his past days.

The dots are the previous days. If you want, you could plot these as lines too. That's what another implementation by github user matkoniecz does:

But Vermeer (following Nick Winter) takes a subset of random points from all of the past days lines, which gives a cloud that conveys where you're generally at throughout the day.

Origin

My understanding is the late Seth Roberts came up with the idea and the name, and thought it helped motivate him to work more and better. He even had some stats showing is daily hours worked before and after he started using it, and how the stats were a lot higher afterward.

My Implementation

I liked this idea when I read about it on Vermeer's site and Nick Winter's book and went about making some improvements:

I hooked mine up to wakatime, to automatically see time spent on various projects.
I made these projects toggleable (both for what went into the green line as well as the background cloud of points) so I could see time spent on certain projects.
I made it all a web app in elm (vs the other main alternative at the time, which was a forked script you had to run locally).
I even registered percentilefeedback.com

I also did a reverse percentile feedback version, for things I didn't want to do (e.g. check twitter or whatsapp). For that I hooked into rescuetime, made things discrete (n of times checked, vs total time), reversed the percentiles (less is better) and made the line red.

Experience

This helped for a bit, and was cool/fine. It was fun to make, and a good opportunity to work on my web app skills.

However, I no longer think things like Percentile Feedback are that effective long term.

Instead, I've come to prefer cultivating habits.

For example, say you have a giant tub of ice cream in your house that you don't want to eat. Having a Percentile Feedback graph is like, "you've only eaten X amount of ice cream today, that's lower than Y% previous days, good job!"

Whereas the habit approach would be to (short term) concentrate and focus your willpower on not eating the ice cream. It's hard at first, but telling yourself you just need to push through it while you're forming the habit helps. After a month (give or take) you'll have the habit, and will be able to abstain with minimal willpower.

The book Superhuman by Habit was influential in my thinking on this, but even before reading it I had stopped checking my charts as often.

]]>

Utopia's Infinite Todo List - Terra Incognita

Thu, 14 Apr 2022 00:00:00 GMT

Utopia's Infinite To-Do List

In Ada Palmer's very unique and excellent series Terra Incognita, the Utopians all wear dark vizors for a futeristic, space helmet vibe.

I enjoyed this passage, where one of the non-Utopian characters, Madame, is trying on Utopian Mushi's vizor:

Madame blew on the inside of the vizor before trying it... “It’s normal!” She exclaimed as the vizor settled in place. ... “The world’s unchanged!” She gazed about at arm, walls, ceiling. “I expected everything would look like [utopia was already here].”
Mushi squinted against the bare air’s unfamiliar cold. “We would hardly work so hard for our utopias if we let ourselves live in the illusion that they are already real.”
She smiled at that. “What are all these floating tags on things?”
“Select one to zoom in. They’re things you can fix or improve: stains or damage, litter, blank walls waiting for art, subjects for research, mysteries, hazards to life or health, clumsy technology waiting for a better alternative.”
She lifted a cup from the nursery table, examining whatever suggestions the vizor made for its improvement. “You see this all the time?”
“Unless I’m watching a movie or something.”
“It would drive me insane. Do all Utopians see this?”
“Almost all. It’s our Infinite To-Do List. It staves off complacency. It’s not easy to maintain a race of vokers in such a comfortable world.”
Madame’s eyes fell on Mushi, dazzled by the calls for medical research and coat improvements which, as she describes it, glittered in the air like fireflies flashing their quick prayers for attention.

Noticing Pain

This reminded me of the part in Ray Dalio's Principles where he mentions -- almost as a throwaway line during a broader discussion of habits -- that his most useful personal habit is reflexively reflecting on any pain he experiences.

When I first read this it seemed almost impossible. Pain happens all the time, is Dalio really reflecting on it anytime it comes up? Since then I’ve tried sort of haphazardly to apply it, and have come to see how it could be powerful.

For example, the other day I was making dinner for the kids and realized I had no idea how much to make. For some reason I thought of Dalio and how this was technically “pain”, and was thinking about what a habit of “reflecting” on it would mean.

The more I thought about it, the more I realized, with this mindset, me not knowing how much food to make (and recognizing it) was really a good thing, because — although I might make too much or not enough this time — now I know how much to use, and don’t need to ever make that mistake again.

I'm still working on cultivating it in myself (another issue: what level of pain am I supposed to be noticing? Are dinner portions getting into diminishing returns?) but a Utopian style infinite todo list helmet would definitely make this easier.

]]>

Fantasy Monday

Tue, 30 Mar 2021 00:00:00 GMT

Fantasy Monday

Fantasy Monday is my free, Monday only subset of Fantasy Math. It lets you put in you, your opponents score, and get back the probability you win (with some nice distribution plots).

It's pretty popular, especially during the first week of the season when there are two Monday night games and everyone's excited about the season. I've written before about Fantasy Math and how I'm not positive it has product-market fit, but Fantasy Monday definitely does. Unfortunately, the price is free.

History

This was actually one of the earliest applications of the model. Way back with Bayesian Fantasy Football, I used to run the matchups for my league every Monday and report back on the league messageboard. The rest of my league has an ambivalent attitiude about my model, but they liked these Monday night updates, and would ask for the probabilities if I hadn't gotten around to posting them.

(While innovative, the Bayesian model had issues, especially towards the tails. I still remember the night a guy in my league won even though the model predicted he had like a 1/100,000 chance to do so — a true Monday Night Miracle!)

Now, of course, my league can check out their probabilities every monday like everyone else at https://fantasymonday.com.

Fantasy Tuesday

During 2020, when a few games got moved to Tuesday due to Covid, I put up a Tuesday night version at https://tuesdaynightmiracle.com. This was more of a marketing gimmick than anything else, but got enough signups to pay for the domain name.

]]>

Quarantine Backyard Bird List

Mon, 08 Mar 2021 00:00:00 GMT

Quarantine Backyard Bird List

When the daycare first closed back in March 2020 my (then) three year old and I decided to start a list of all the birds we saw in our yard during quarantine. We were in the middle of Spring migration and I figured it'd be interesting to keep track of what we saw over the next month or so.

...

Almost one year later and the kids are about ready to go back to daycare, which means it might finally be time to call it a wrap on the list.

Here's what we got:

Of all of these, I think most interesting might be the Nashville Warbler, which I never would have been able to identify had it not unfortunately hit one of our windows.

The Rose Breasted Grosbeak were also interesting and easily identifiable. They came through on a migration and stayed exactly one day.

All in all, I'd say the kids were moderately interested. Penelope can recognize probably half the birds on here, and likes to take a look through the binoculars at one every now and then.

]]>

A Simple Explanation of the Monty Hall Problem

Mon, 01 Feb 2021 00:00:00 GMT

A Simple Explanation of the Monty Hall Problem

Background

The Monty Hall problem — based off of the TV Show "Let's Make a Deal" and named after the original host, Monty Hall — is a notorious problem in statistics.

Problem

Monty presents to you three closed doors. Behind one is a prize, behind the other two are nothing (I think the original formulation says they're goats — either way, not something you want).

You pick one of the doors.

No matter which you pick, Monty (who knows what's behind every door) opens an empty door, and offers to let you switch your guess to the other, remaining closed door.

Should you switch? Does it matter?

Answer, which humans are bad at understanding

You should indeed switch. Your odds of winning are 2/3 if you do, vs 1/3 if you don't. However, many people find this difficult to grasp at first.

Jeff Kaufman has a post where he talks getting it wrong initially, and how bad humans are at it in general.

"The person who first showed me this problem failed to convince me that I should switch; that took testing it with pennies and cups. ...

"Since then, in various conversations with people who were sure switching didn't help, I've tried many times to describe how the odds for switching can be 2/3. I've come up with many explanations that I think would have convinced me, but they don't seem to convince others."

He's not alone:

"You should switch. However, it is so counterintuitive that I had to write a simulation script with 1M iterations... to confirm indeed switching yields twice the probability of winning than staying." — u/creekwise

"I also wrote a simulation script." — u/Jusque

Simple Explanation

Here's the clearest way I've found to think about it.

Say you always switch. Now there are two outcomes:

You originally pick one of the two dud doors (2/3 chance). Then Monty will open the other dud (he has to, the only other option is the prize door and he can't open that) and so if you switch to the unopened door, you'll win.

You originally pick the prize (1/3 chance). Monty will open one of the two duds, you'll switch to the other, unopened dud. You won't win.

So: 2/3 of winning if you switch. Don't switch, it all depends on whether you pick the prize initially, which you'll do 1/3 of the time.

]]>

Vim For Everyone

Fri, 15 Jan 2021 00:00:00 GMT

Vim For Everyone

Vim For Everyone — at vimforeveryone.com — is a work in progress that I put up early in 2021 to extol the benefits of Vim for non programmers.

What is Vim?

Vim is a really fast text editor. Most people use it to edit code, but it's also great for maintaining your own personal knowledge base (wiki, notes to yourself, etc), which is the use case I'm advocating on the site.

The main motivation to use Vim is speed. Going from hunting and pecking to regular touch typing is big jump. Going from regular typing to Vim is a similar jump.

I'll save the evangelizing for the site, but suffice to say if you're interested in taking notes on, and ingesting and making connections between information — and you like keyboard shortcuts — you should definitely check out Vim.

Motivation

I've been using Vim for about 10 years. Initially, it was for code, but within a few months typing in anything else was too slow and frustrating. Since then I've been using Vim to take and manage all my notes (what I'm reading, meetings, to plan my days, etc).

I was inspired to create Vim For Everyone after hearing a lot of hype around other personal note taking tools — Roam, Notion, Workflowy, Evernote etc. After looking into them, it was clear (at least for me) they weren't going match the setup I had in Vim.

Market Research

Out of curiosity, I poked around to see if there was a thriving community of non technical Vim users (like the Roam cult but for Vim) and found... exactly one reddit post from 2015 with 38 upvotes.

Hence the site. Vim isn't literally for everyone, but the non-coder Vim space is criminally underdeveloped.

Goals

The purpose is more to get non-coders intrigued by Vim than to be necessarily definitive source for learning how to use it.

Part of the issue is I learned it ~10 years ago and haven't thought recently about the best way to get it across. However, I will probably write up some thoughts at some point.

Bottom Line

Here's how useful the non-coding, note-taking aspects of Vim has been for me: if Vim For Everyone gets a few new people to try Vim, and a handful of them get as much out of it as I do, it'd probably be the most valuable thing I've built to date.

]]>

Notes on Arnold Kling's Three Languages of Politics

Wed, 13 Jan 2021 00:00:00 GMT

The Three Languages of Politics

One of the simplest, most illuminating books I've ever read is Arnold Kling's The Three Languages of Politics.

Kling's idea: conservatives, liberals and libertarians communicate (more on whether it's just communicate or something else later) about political issues on three, separate axes:

progressives on the oppressed vs oppressor axis
conservatives on the civilization vs barbarism axis
libertarians on the freedom vs coercion axis

Once you notice this framework, it becomes a lot easier to understand other's views on almost any issue.

Many issues span all three axes. Illegal immigration is typical example.

Progressives view immigrants as oppressed by unjust immigration laws, border patrol agents, ICE, etc. They're more likely to pay attention to kids in cages or stories about fully assimilated DREAMERs excelling in school at risk of being deported.

Conservatives argue that immigrants (particularly illegal, but legal too) have negative cultural impacts on "civilization" (western, market oriented, traditional values) at the margin.

They also tend to focus on the fact that (by definition) illegal immigration is against the law. They talk a lot about rates of criminality by immigrants, and pay attention to anecdotes about immigrants advocating things like Sharia law.

Libertarians believe people should have the freedom to live, work and enter whatever mutually beneficial economic arrangements they want. They are against immigration laws that prohibit this.

They'd pay more attention to stories of hardworking immigrations who came up from nothing (conservatives also appreciate this narrative) who are driving a taxi to earn money to send back to their families.

Other Examples

Other examples of issues that pretty clearly span all three axes (see if you can mentally make the argument for each axis):

drug laws
abortion
soda taxes
whether wedding cake bakers can discriminate against gay couples

(The last two come directly from Kling's book).

If I were a middle or high school social studies teacher, I'd make students analyze issues in terms of the axes all day long.

Some issues are fights between two axes

Many issues have three distinct axes based arguments. But often a fight about an issue will be between two of the three camps. Take California Proposition 22, which is about whether Uber and Lyft should be able to classify their drivers as independent contractors instead of employees.

Progressives argue classifying drivers as contractors is a way for ride sharing companies to exploit drivers. They think Uber is oppressing drivers by not paying mandated employee benefits (overtime, paid sick time, health care, etc).

Libertarians argue out the rideshare-driver relationship is between the companies and their drivers. No one is forced to drive for Uber. If someone needs healthcare, paid sick leave, etc, they're welcome to find a job that offers it. If enough people feel similarly, to the point Uber was having trouble attracting drivers, they'd change their business model.

This particular fight doesn't seem to fit the conservative axis all that well.

Axes aren't always equal

On some issues, a dominant axis may emerge, where most people find themselves on a certain axis, even if they don't usually frame issues this way.

For example, most people today view 1960s civil rights struggle in US on the oppressor-oppressed dimension. This clearly wasn't the case at the time; many people then thought individual businesses and people should have the freedom to discriminate if they want.

While some people still believe this, they're pretty hardcore on the freedom-coercion axis. Most people in US today view the issue through the oppressed-oppressor lens, and take a favorable view of civil rights.

Other examples: initially, I think most people viewed both 9/11 and the Charlie Hebdo shootings in civilization-barbarism terms (though apparently many on the left have recently been accusing Charlie Hebdo of racism, so maybe not).

Extensions and Open Questions

Having though about the three axis model for a while and gotten in the habit of viewing contentious issues through the various lenses, here are some miscellaneous thoughts and open questions.

1. It's not that people are just on different axes, they often take issue with the "good" ends of other axes

Each axis has its good end, i.e. freedom, civilization, non-oppression (OK, it's debatable whether the progressive axis has a "good" end or whether they're just anti-oppression).

It seems like thinking primarily on an axis means you tend to discount whether the good ends of the other axes are worth pursuing. I think helps explain certain behavior, and makes the model even more useful.

For example, conservatives care about civilization. Do progressives and libertarians?

Progressives and civilization. I think anti-civilization feelings on the left are most associated with the environmental movement. You can also see it in tendencies to romanticize pre-colonial, indigenous societies.

Libertarians and civilization. I would say libertarians appreciate civilization more than progressives, but not as much as conservatives. Example: many libertarians advocate open borders on principle, even if it might put some of the freedom/market aspects of western culture at risk.

Libertarians care about freedom. Do progressives and conservatives?

Progressives and freedom. There is obvious, historical evidence that progressives don't value freedom that highly. But even today, at the margin — are progressives in favor of individuals being free to spend their money on (or hire, or eat, or wear) what they want, or less?

Conservatives and freedom. Again, conservatives probably care more about freedom than than progressives, but they're definitely willing to make it take a back seat if they think other things are more important. See the NSA, Patriot Act, Snowden, drug war, historical opposition to gay marriage, etc.

Progressives care about stopping oppression. What about conservatives and libertarians?

Conservatives and oppression. While very few people think of themselves as pro-oppression, I think many non-progressives take issue with much that is categorized as oppression today. For example, conservatives point out that when you control for violent crime rates, it doesn't appear minorities are systematically more likely to be shot by police than whites.

Libertarians and oppression. I think libertarians would generally agree (though on this issue, many would also say police are too militarized and powerful). They also have more of a freedom bent, and are hesitant to call anything the result of voluntary transactions oppression. "Yeah, working in a sweat shop would suck, but these workers technically have a choice".

While we're on the subject of police shootings, an aside: the dueling progressive-conservative slogans, "black lives matter" vs police as the "thin blue line" that protect us from anarchy are straight out of the three axes model.

2. Is this just the way people communicate, or is it something more?

Kling is careful to say he's talking about communicating about political issues. That's why the book is called the three languages of politics.

"Let me quickly add that I do not believe that the three-axes model serves to explain or describe the different political ideologies. I am not trying to say that political beliefs are caused by one's choice of axis. Nor am I saying that people think exclusively in terms of their preferred axis.

I'm not sure that goes far enough, though it's obviously noisy. But I would expect, for example, that people who identify with the freedom-coercion axis generally favor smaller government, fewer regulations etc.

Also, given the way most people's internal monologue, language as thinking brains work, the difference between language and thinking isn't necessarily clear.

3. Is there another axis?

In a follow up edition, Kling toyed with adding what he called a bobo/anti bobo axis.

Bobo is an idea from the NYT columnist David Brooks 2000 book Bobos in Paradise, which (in my understanding, I've never read the it) is basically the cosmopolitan/elite/smug/"believe science" vs the backlash/anti-smug/troll/"the CDC originally said not to wear masks" axis.

In particular, the phenomena of Trump, who — while he definitely has conservative themes like law and order, isn't exactly the epitome of western civilization — makes this axis intriguing.

4. On average, are members of some groups at more "extreme" ends of their axis than others?

Here I'm mostly thinking of progressives, which seems to have taken the oppression/non-oppression axis so far in one direction (particularly when it comes to racial, sexual and gender identities) that it always reminds me (not sure if this is weird) of allergies.

Allergies are caused by your body's immune system reacting too strongly to things that are really not that big of a deal. Similarly, the left's hyper sensitivity to any sort of (real or perceived) identity-based oppression is causing real dysfunction, both within the left itself and in society at large.

5. Many of today's ideological fault lines seem to be about nature vs nurture, how does this fit in?

My guess: this is mostly relevant in impacting how people view the extent of oppression.

Take the underrepresentation of women in tech. If everyone is a Blank Slate, the fact there are fewer women in tech might be due to oppression.

But if — on average (individuals are all over the place, which is good!) — women are naturally less inclined to study tech related fields (whether because of a things/people dynamic or because on average girls are better at reading, whatever) than this disparity might be just something that happens, no oppression necessary.

Note: even if true, this doesn't at all mean women can't like tech or are "weird" (or less feminine) if they do, any more than men who like to read. Again, differences within groups swamp differences between groups. It also doesn't preclude oppression.

6. How does this theory fit with with Bryan Caplan's simplistic model of left and right?

Caplan's theory: the left hates markets, the right hates the left.

While possibly tongue in cheek, these models do attempt to explain similar things.

7. Where do conspiracy theories fit?

Are certain groups (OK, conservatives) more inclined towards conspiracy theories? If so, why?

8. How much do these axes bleed into each other?

When you stare at them for a while, the boundaries between axes gets a bit fuzzy.

oppression is arguably another word for infringing on someones liberty
many progressives frame their oppression arguments in terms of "positive liberty", e.g. if a sweatshop worker or a poor kid in the ghetto doesn't have good options, they're not technically free.
one of the foundational concepts of western civilization is freedom (life, liberty, pursuit of happiness) and equality under the law

So, question: to the extent there is overlap, what should we do about it?

Answer: probably just not think about it too hard. It's a model, not necessarily exactly the way the world works in granular detail.

9. Do people's cobbled-together, irrational, self-justifying minds make thinking about axes less useful?

E.g. see the GOP House Freedom Caucus, which starting in 2016 did a massive shift away from Freedom/towards Trumpism. To the extent this sort of thing happens (both at the individual and group level), the model seems less useful.

]]>

Learn to Code with Baseball Writeup

Sun, 22 Nov 2020 00:00:00 GMT

Learn to Code with Baseball

Learn to Code with Baseball — available at codebaseball.com — is the baseball adaptation of Learn to Code With Fantasy Football.

I wrote it early in 2020, put it down for a while due to Covid, and picked it back up in the fall.

Like the football version, it covers Python, Pandas (Python's main data analysis library), SQL, web scraping, public APIs, data visualization, summary stats and modeling.

Writing Process

Because the original football version is much more focused on teaching programming and data science basics, which are similar across domains (including non sports domains!), porting the same concepts to baseball was fairly straightforward.

Baseball is much more stat heavy than football, which helped. I personally don't follow it as much as football, which didn't help. But I did some research, and I do like reading baseball books (Moneyball is an all time favorite, also really liked The Only Rule is It Has to Work).

One difference is that baseball is much more focused on good composite stats and measuring individual vs (fantasy) football, and the analysis portion of the book reflects that.

]]>

Intermediate Coding with Fantasy Football Writeup

Thu, 12 Nov 2020 00:00:00 GMT

Intermediate Coding with Fantasy Football: A Project Based Guide

Intermediate Coding with Fantasy Football: A Project Based Guide — available at fantasycoding.com/next — is a follow up to Learn to Code with Fantasy Football I wrote mostly over the Summer of 2020.

It walks readers (who it assumes already know basic Python and Pandas) through building three projects:

Who Do I Start Calculator
League Analyzer
Best Ball Projector

All of these projects are built around the Fantasy Math API, access to which is included in the package (also includes access to the GUI version of Fantasy Math too).

Motivation

While the reactions to Learn to Code with Fantasy Football have been positive and many people have built cool things after reading it, I had heard from a few people that they weren't quite sure where to go or what to do next.

In particular LTCWFF didn't go that in depth on the process of developing real life programs. I figured this was an opportunity to do that.

All three projects are built around GraphQL API access to the newly rebuilt (in PyMC3) Fantasy Math model, which I continue to believe is underrated.

I figured this was an opportunity to expose LTCWFF readers to the Fantasy Math model, possibly even one they'd be willing to purchase annually.

Reception

Reception among people who have bought it has been good, there just haven't been as many buyers as I'd have liked.

One issue is that LTCWFF is pretty comprehensive, and a few former readers I reached out to about this follow up already considered themselves intermediate and were lukewarm about the idea ("sounds cool, I'll probably buy it if it's not too expensive"). So I think it's likely most of an issue of market.

Reflection

Overall, I'd say this ended up being not worth my time to do, but there were some positives. For one, I ended up hosting the GraphQL API myself (on a DigitalOcean box) and learned a decent amount about hosting, deploying and dev op type stuff from doing that.

And now that it's done, it shouldn't be that hard to spin up in future years if I decide to do that.

]]>

Teaching Kids About Randomness

Wed, 21 Oct 2020 00:00:00 GMT

Teaching Little Kids About Randomness

The other day my 4 year old and I had to decide between two options and got on the subject of flipping coins. I thought it could be a good opportunity to learn about randomness, so we invented the following "game".

How You Play

Find a quarter (this is the hardest part of the game).
Everybody writes down their guess, heads or tails.
The kid flips the quarter. Initially Penelope's flip was more of a drop and it was debatable whether we were getting suficient entropy, but eventually she developed her own technique of bouncing the coin off the wall, which worked.
Record the results.
Do steps 2-4 ten times.
Tally the results, both who guessed the most correctly as well as the overall number of heads and tails.

Notes

Yeah, this "game" is 100% luck, but most little kid games are (war, candyland etc).

The benefit of doing 10 flips and recording them (as opposed to just flipping a coin a bunch of times) is it let us see the breakdown of heads and tails at the end, along with the order, both of which help give a more intuitive feel for randomness.

I thought about leaving it at that, just flipping the coin 10 times and recording the results, but I think writing down everyone's guess before each flip helped Penelope stay engaged. And she definitely was engaged, she remembered everyone's guesses every time and called out who was right and wrong even before the coin stopped spinning.

We've done it just the one time so far, but we'll keep track of the results if we do it again. Enough times and she might have a good grasp of the central limit theorem before she's 5.

Our results. (P)enelope is my 4 year old, (D)ad is me. (G)randpa and (A)rlo (her two year old brother, who sort of guessed but definitely did not get what was going on and also scribbled on the sheet) also played. Can see Penelope is the best guesser!

]]>

Keeper + Draft Pick Trade Calculator Writeup

Tue, 22 Sep 2020 00:00:00 GMT

Keeper Trade Calculator

The Keeper Trade Calculator was a tool I built along with the Draft Pick Trade Calculator in 2015.

See that write up for more on the method.

The TLDR: use past average auction values (AAV) to see the expected AAV at each pick, use that.

Adding in Keepers

Once you do that, adding keepers is a natural extension, just use the player's average auction value and compare it to the value in draft picks.

Of course, keeping players makes draft picks less valuable (they're effectively later) but that's easily adjustable. Just have the calculator effectively move the picks back depending on who's kept. This meant anyone using the calculator to enter in all the league's keepers.

Extensions

In my main league we just keep one player and that's it, but I know a lot of teams lose the pick in the round the keeper was picked in.

In theory, AAV makes it easy to evaluate these trades too, and you could actually hack the calculator to do decent job at it.

Caveats

Same as the Draft Pick Trade Calculator version. The biggest problem is the MFL auction data is so thin. I'd really like to figure out a good way to get updated, flexible average auction values.

Tech

This was one of my first projects in React. I hosted it in a $5 droplet on Digital Ocean using Dokku. In hindsight that was completely unnecessary since everything was stored client side, but I didn't know much about web development then.

]]>

Notes on Deep Work by Cal Newport

Wed, 09 Sep 2020 00:00:00 GMT

Notes on Deep Work

Definitions

Deep work: professional activities performed in a state of distraction-free concentration that push your cognitive capabilities to their limit, create new value, improve your skill, hard to replicate

As opposed to shallow work: non-cognitively demanding, logistic style tasks, often performed while distracted, low value and easy to replicate.

Most workers (esp knowledge workers) do a ton of shallow work.

There is a massive opportunity for people who prioritize depth.

Part 1: Motivation

The first section of the book is Newport making the case that deep work is valuable, rare, and meaningful.

Valuable

It's essential to (1) learn new, hard skills quickly and (2) perform at a high level (both in terms of speed, quality) both involve pushing skills and capabilities to their limits.

Newport also talks about biological/neurological mechanisms for why learning/working while distracted is less effective

Rare

Newport basically needs this chapter so people can't say, "if deep work is so valuable, why don't more people do it?"

His answer: (1) metric black hole (hard for businesses to measure who, what is really productive) and (2) path of least resistance (running day out of inbox, sitting on skype just easier) another reason: culturally, people view tech as being worth it if it provides ANY benefit (regardless of costs).

For example: "I read that really cool thing on twitter once" vs the usual where you wasted a bunch of time/were distracted.

"Principle of least resistance + metric black hole = work cultures that save us from short term discomfort of concentration and planning at the expense of long-term satisfaction and creation of real value." - Newport

Meaningful

Happiness researchers: "best moments usually occur when a person's body or mind is stretched to its limits in a voluntary effort to accomplish something difficult and worthwhile."

Most people underestimate happiness at work, overestimate happiness relaxing.

Much of shallow work is trivial, annoying, stressful <- not deep working means this is at the forefront of your attention much of the time.

Part 2: Practice

The 2nd part of the book is HOW to do more/better deep work necessary because deep work isn't just something that'll happen because you want it to, you need systems, routines, habits, etc.

He supports with examples from historical, known deep workers, lessons drawn.

Bottom line basically: don't be haphazard in work habits don't wait for inspiration.

In talking about how to do deep work Newport zooms in and out of different levels over different chapters.

Generally, I'd divide the advice up as:

High level: identify what you're deep working towards, track deep work, do weekly reviews.

Lower or medium level, e.g.:

plan out every minute of your day
be done working at a certain time
say no/minimize shallow obligations

Separately, Newport puts a big emphasis on -- even when not (deep) working -- not giving in to mindless phone/internet/distraction

High Level.

Newport goes through some business literature talking about HOW people achieve goals.

identify small number of ambitious outcomes to pursue with deep work hours
act on lead measures (time spent in a state of deep work dedicated towards ambitious outcomes)
keep a compelling scoreboard (e.g. tally of hours deep worked per week in office)
create a cadence of accountability

For (4), Newport did weekly reviews, where he look over scoreboard and celebrate good weeks, understand what led to bad weeks, figure out how to ensure good scores in days ahead

Lower/Medium Level Planning Daily Planning

Most people spend more time on shallow work then they realize.

To combat: plan every minute of your day in 30 minute increments.

Cal newport does his daily planning either: (1) night before, (2) first thing in the morning, (3) after he knocks out a big task in the morning.

Don't just plan work, plan lunch, relaxation too.

Flexiblity

Planning doesn't mean you can't be flexible, if you get interrupted or things take longer just replan remainder of day when you need to.

The goal isn't to stick to a given schedule at all costs; it's instead to maintain, at all times, a thoughtful say in what you're doing with your time going forward -- even if those decisions are reworked again and again as the day unfolds.

Can plan "task blocks", where just knock out smaller, specific things you have to do.

Newport has a rule for himself: if stumbles upon a really important insight, OK to spend rest of day thinking about it.

Most people use a plan as wishful thinking at first (definitely true in my experience), but should try to be accurate/conservative.

Weekly Planning

Note: not a lot of stuff on weekly planning in Newport's book, instead some of these notes are from posts on his blog. usually spends 1+ hour planning week

Be flexible, can do goals for each day or broader heuristics ("need to spend 3 hours each day on...").

Big part of weekly planning process if working back from calendar to fill open time effectively.

Full Horizon Planning

Same as above about not in Deep Work, found on blog.

Newport views work in terms of projects. Each project has two states: dormant or active.

If it's dormant, tracked somewhere that he regularly reviews (so he won't forget about it).

If it's active, he makes a plan for how and when the whole thing will be completed.

"To pull this off, this plan must exist at multiple levels of refining granularity. That is, on the monthly level, I know what weeks I will work on the project, and only when I get to those weeks do I plan out what days I will work on it, and only when I get to the specific days do I figure out which hours it will consume."

"These plans, of course, change as things unfold, but the point is that I don’t deal in abstractions, I like to work directly with the brute physicality of time. This makes sure I get the most out of the cycles I have available, and it prevents me from committing to more than is feasible."

Be done by 5:30

Newport calls this "fixed-schedule productivity"

Works backward to fit all he needs to get done in his time allocated for working. He thinks it makes him more productive.

"Because my time is limited each day, I cannot afford to allow a large deadline to creep up on me, or a morning to be wasted on something trivial, because I didn't take a moment to craft a smart plan."

Includes some non-work benefits:

more present with family
reads more books
more comfortable being bored

Eliminate shallow obligations

You should get comfortable saying no to things. No should be the default.

"The bar for gaining access to your attention and time should be very high."

Email

Do more work when sending/responding.

It's tempting to do quickest thing to get it out of your inbox, but wastes time in the long run. Instead think about "project" the email represents, what the desired state of that project is and how to get there, then in your response make clear where you are currently/how to get there.

Get comfortable not responding. Don't respond if:

it's not something that interests you
nothing really good/bad would happen if you did/didn't respond

Distraction

Obviously distraction during work is bad, it's one of definitions of shallow work.

But even when you're not working, will be much better off if can wean self from dependence on distraction

"Same way athletes take care of bodies outside just training, you'll struggle to achieve the deepest levels of concentration if you spend the rest of your time fleeing the slightest hint of boredom"

I.e. don't just give in and mindlessly surf when you're waiting in line, etc need to be comfortable being bored.

This will help you work even deeper when you are doing it

Instead of being always online, on phone etc, with occasional breaks to focus, do the reverse: always off, with scheduled times to give into distraction if you actually need to use the internet a lot, just schedule more time for it

This isn't in deep work, but this helped me re phone (basically solved me feeling like I spend too much time on my phone, wish every problem I had was this easy to solve):

https://maketime.blog/article/the-distraction-free-iphone/ https://maketime.blog/article/six-years-with-a-distraction-free-iphone/

]]>

Scramble Tracker

Mon, 24 Aug 2020 00:00:00 GMT

Writeup of scramble tracker project.

Notes on Principles by Ray Dalio

Tue, 18 Aug 2020 00:00:00 GMT

Notes on Principles by Ray Dalio

Principles by Ray Dalio is one of my favorite books. I think everyone would benefit from reading it.

The book is divided into three parts:

Dalio's autobiography (ok, but not earth shattering)
Life principles (excellent)
Work principles (the life principles applied to work, repetitive and definitely less practical).

To start, I'd just read part 2. That's what Derek Sivers, who also really liked the book — "I wanted to highlight almost every paragraph" — and even Dalio himself at one point recommend.

Examples

This book is good, but can give off a platitude vibe, which turns some people off.

So I figured I'd give a few examples of ideas that I've personally found useful. If these seem intriguing, might want to check out the rest of my notes (further below) or read the book.

Navigating Levels

One idea that was useful to me when writing LTCWFF was the idea of "levels".

Reality exists at different levels and each of them gives you different but valuable perspectives. ... We are constantly seeing things at different levels and navigating between them, whether we know it or not, whether we do it well or not... For example, you can navigate levels to move from your values to what you do to realize them on a day-to-day basis.

> I want meaningful work that's full of learning.
>> I want to be a doctor.
>>> I need to go to medical school.
>>>> I need to get good grades in the sciences.
>>>>> I need to stay home tonight and study.

According to Dalio, when a line of reason has gotten jumbled, it's often because the speaker is jumping around different levels without showing how they connect.

That's why, when conveying information, it's important to:

Be aware what level you're examining a given subject.
Consciously navigate levels rather than see subjects as undifferentiated piles of facts that can be browsed randomly.

Of course, I had always known that in theory, but explicitly keeping it in mind has probably made my thinking and writing better.

Believability

Believable people are those who have repeatedly and accomplished the thing in question — strong track record of at least 3 successes — and have good, logical explanations of how they did it when probed.

If you have a different view with someone who is more believable than you, you should make clear you're trying to understand their perspective.

Designer vs Worker Level You.

Dalio recommends thinking of yourself as a worker operating in a machine that you've designed, and says it's important to distinguish between you as the designer of your machine, and you as a worker within your machine.

"It's much more important that you are a good designer/manager of your life than a good worker in it."

That's not to say being a good worker isn't important (Dalio also says good work habits are way underrated), but it's not as important.

"To be successful, the "designer/manager you" has to be objective about what the "worker you" is really like, not believing in him more than he deservers, or putting him in jobs he shouldn't be in. Instead of having this strategic perspective, most people operate emotionally and in the moment; their lives are a series of undirected emotional experiences, going from one thing to the next."

This is one of my favorite concepts, one I've found particularly useful.

For example: for a long time, I had been getting up early (and staying up late) to hack away on side projects (which I always hoped would eventually take off) but found once we had our second child that that was really hard to do.

So I decided to go down to 80% at my day job to spend one day a week working on my own stuff. I viewed this as a "designer level you" move, vs just continuing to hack along harder and harder (and with less and less sleep) in my role as "worker me". My first project made more than 20% of my old salary, and now I'm working on my things full time.

General Notes & Questions

See full notes below for more info/summary.

Connections between chapters?

Navigating levels principle (5.4) sort of ironic, because (though principles are all good) sort of feels like we're jumping levels here, particularly from chapter to chapter.

One possibility: starts with 1 (embrace reality and deal with it) by (going one level lower) doing 2 (5 step process).

Then chapters 3-5 (open-mindedness, wired, decisions) are a way to do 2 (the five step process) better.

Themes

Could potentially reorganize around themes, for example:

higher vs lower level you
embracing reality
you are designer and worker in your machine
open minded/pain as training
believability
navigating levels and synthesizing

Favorites

My favorite principles/concepts:

look at your machine from a higher level (1.10)
being realistic about your own weaknesses (1.10)
higher level vs lower level you (3.1, 4, 5.1) and habits (4)
navigating levels (5.4)
seek out pain to grow (1.7) and id and don't tolerate problems (2.2)
designing plans with timelines and metrics (2.4)

Not as new to me personally but def critical:

own your outcomes (1.9)
decisions as EV calculations (5.6)

Issues

What to do when you don't have the type of access to experts etc Dalio is talking about?

It's one thing for Ray Dalio, who can talk to Bill Gates or the Dali Lama, but what am I believable at and what believable people do I have access to?

Full Notes

Full notes start here.

Dalio thinks of life where each problem is a puzzle. The reward is principle that helps avoid same sort of problem in future. He's written down the principles that've worked for him and put them into this book.

1. Embrace Reality and Deal with It

Nothing more important than understanding how reality works and how to deal with it.

1.1 Be a hyperrealist

dreams + reality + determination = successful life

1.2 Truth (accurate understanding of reality) foundation for good outcome

Most people fight seeing what's true when it's not what they want.

It's more important to understand/deal with bad stuff because the good stuff will take care of itself.

1.3 Be radically open-minded and radically transparent

Both are invaluable for rapid learning and effective change.

They make what you are doing, and why, clear to yourself and others.

more open-minded: less likely to deceive yourself
ability to reflect on inevitable feedback requires open-mind

Being radically transparent easier the more you do it. It's not about talking about everyone's secrets, but being transparent with opinions of each other and how world works.

Both bring more meaningful work and meaningful relationships.

1.4 Look to nature to learn how reality works

1.5 Evolving is the point

1.6 Understand nature's practical lessons

Dalio spends some time talking about how nature "evolves for the good of the whole", which I'm not sure I really buy. I personally think nature/the universe is pretty indifferent to what's "good".

But I think Dalio is basically saying: learning, progressing, working towards goals is point/what will give you the most satisfaction out of life. I do agree with this

It's nice to think about because Dalio has accomplished so much, if he's still improving and happy doing it at $18B or whatever, pretty much should work for anyone.

Dalio: thrilled to be infinitesimally small part of whole; finds reality and how nature works beautiful. Goal is to simply evolve and contribute in some tiny way while he's here and is what he is.

One of nature's practical lessons: adaptation through rapid trial and error invaluable.

Realize you're everything and nothing: everything to yourself, and nothing in the scheme of things.

It is a reality that each one of us is only one of about seven billion of our species alive today and that our species is only one of about 10 million species on our planet. Earth is just one of about 100 billion planets in our galaxy, which is just one of about two trillion galaxies in the universe. And our lifetimes are only about 1/3k of humanity's existence, which itself is only 1/20k billion of Earth's existence."

Success

Dalio defines success as struggling and evolving as effectively as possible.

In other words, learning rapidly about oneself and environment, then changing to improve.

"Love and work are the cornerstones of our humanness." - Freud

Your work doesn't have to be a job, but Dalio thinks it's better if it is.

1.7 Pain + Reflection = Progress

A fundamental law of nature: in order to gain strength, have to push limits, which is painful.

This is especially true about confronting harsh reality of own imperfections.

No pain no gain.

Stuff won't sustain happiness.

There's no avoiding pain, esp w/ ambitious goals.

Developing reflexive reaction to pain that causes you to reflect on it rather than avoid it will lead to rapid learning/evolving. Later in book Dalio says this is his most impactful habit

aka face the painful realities caused by problems, mistakes, weaknesses

Most people have hard time reflecting while in pain and pay attention to other things when pain passes. But at a minimum, you should try to remember to reflect after it passes.

You're not maximizing potential unless you are:

pushing limits
occasionally failing
sometimes breaking through

You get benefits from both failing and breaking through. If you do the whole process successfully, you will keep ascending in life (Dalio has a loop like drawing he likes).

It gets easier over time. First, you realize things seem bigger than they are when seeing up close.

Also, higher you ascend, more you realize most things are "just another one of those".

Go to pain rather than avoiding it.

You'll evolve faster when comfortable operating at some level of pain.

The pain is the signal/what you need to seek out; like exercising.

When you're do it right, you'll be glad to:

identify, accept, learn how to deal with weaknesses
have people around you be honest with you vs keeping negative thoughts about you to themselves
be yourself vs pretending to be strong about something you're weak at

Note this loop/feedback/operating at pain thing isn't just a matter of dealing with success (seeking out harder and harder goals), but major failures too, which will happen.

1.8 Weigh 2nd and 3rd order consequences

People who overweigh 1st order consequences rarely reach their goals.

Example: temptations like shitty food or avoiding pain from exercise.

1.9 Own your outcomes

Mostly, life gives you so many decisions to make and so many opportunities to recover from mistakes that you can have a great life.

Whatever circumstances are, will be more likely to succeed and find happiness if you take responsibility for making decisions well.

1.10 Look at machine from higher level

Think of yourself as a machine operating within a machine; and know you have ability to alter machines to produce better outcomes.

Distinguish between you as the designer of your machine, and you as a worker with your machine.

It's much more important that you are a good designer/manager of your life than a good worker in it.

"To be successful, the "designer/manager you" has to be objective about what the "worker you" is really like, not believing in him more than he deservers, or putting him in jobs he shouldn't be in. Instead of having this strategic perspective, most people operate emotionally and in the moment; their lives are a series of undirected emotional experiences, going from one thing to the next."

Get over ego driven emotions re: people watching you struggle (anger, embarrassment, defensiveness, etc.

Weaknesses

You have four choices when encountering weaknesses:

Deny them (most people do this).
Accept them and work at them to convert into strengths (may or may not work, prob best option if you can do it and it's worth it).
Accept them and find ways around them (very under utilized).
Go after something else.

If you're open-minded and determined, you can get pretty much whatever you want.

No matter what you're like, many paths will be suitable. If a particular path closes, just need to find another good one

In the continuum of savoring life/making an impact, dalio wanted both.

Getting more out of life not just a matter of working harder at it, it's a matter of working effectively.

Have to decide to what extent you will put interests of others above your own, and which others. Will do either consciously or subconsciously, so should think about it.

2. Use 5-Step Process to Get What You Want Out of Life

Have clear goals.
Identify, don't tolerate problems that stand in the way.
Accurately diagnose problems to get at root cause.
Design plans that will get you around then.
Do what's nec to push designs through to results.

Have to do this fast, continuously and iteratively, setting goals higher and higher.

Import to do all of these one at a time and in order.

2.1 Have clear goals

Have to prioritize, but good news is can have much more than nec to be happy.

goals != desires

Desires are things you want (like junk food or sleeping in) that can prevent you from reaching goals.

Don't rule out a goal because you think it's unattainable; if you're limiting your goals to what you know you can achieve that means the bar is way too low.

Need: flexibility, self-accountability.

flexibility: accepting what reality teaches you
self-accountability: take failure to achieve as personal failure

Note: not always about moving forward, dealing well with setbacks/tragedy can be goal too.

2.2 Identify and don't tolerate problems in your way

Each problem an opportunity, essential to bring to surface

even if unpleasant
related: go to pain, don't avoid it

Don't mistake cause of problem with problem.

Example: can't get enough sleep (potential cause) vs performing poorly in job (the problem).

Be specific and precise in identifying them.

Prioritize and work on larger, important (to you) problems.

Once you identify a problem, don't tolerate it.

2.3 Diagnose problems to get at root causes

Focus on what is before what to do about it.

Moving too quickly to a solution is common mistake.

Root causes manifest themselves over and over again in problems.

Distinguish proximate causes (verbs) from root causes (adjectives)

"I missed the train because I didn't check the train schedule."
"I didn't check the train schedule because I am forgetful."

Can only truly solve problems by removing root causes.

2.4 Design a plan

Replay how you got to this point, then figure out what you have to do going forward.

Think about outcomes produced by some machine, think how to change your machine (again thinking about worker you and designer you).

Remember there are likely many paths to achieving goals; just need to find one.

Think of things like a movie script, visualizing who will do what through time.

Should go from big picture to details w/ timelines.

Write down plan to see and measure progress against, including tasks. Remember tasks, narrative, goals all different.

Making a good plan doesn't take that long, but too many people don't do it at all.

2.5 Push through to completion

Great planners who don't execute go no where.

Important to remember the connections between tasks and goals they are meant to achieve. Don't lose sight of the "why".

Good work habits are vastly underrated.

Establish clear metrics to make sure you're following plan. If you're not hitting targets, that's another problem to diagnose and solve.

Note the process is iterative.

when you complete one step it'll likely lead you to modify others
when you've done all five, you start again with a new goal
if process is working, goals change more slowly than designs, than tasks

Process is both synthesizing (first three steps) and shaping (designing solutions and implementing).

2.6 Remember weaknesses don't matter if you find solutions

You probably can't do all of these steps well.

Good thing to do: rely on others.

Everyone has at least one of these that they're not that great at that stands in the way of success, figure out which it is for you and deal with it .

3. Be Radically Open Minded

3.1 Recognize your two barriers

Two biggest barriers to good decision making are ego and blind spots.

Make it hard to objectively see what is true about you and your circumstances and make best possible decisions by getting most out of others.

Higher level vs lower level you

Have two "yous" that fight to control you

Lower level

impulsive, emotional, subconscious
deals with ego/defense mechanisms re: criticism etc

Higher level

neocortex
more logical, executive fn

Can see it w/ people getting angry for themselves.

Even more of a mess when two yous fight with someone else with their yous. Lower levels are like attack dogs even when higher levels want to figure things out

Blind spot barrier is more in terms of ways we think/view things, what we might miss.

Both lead to people disagreeing and people remaining convinced they're right.

not logical and leads to suboptimal decision making
if you disagree with someone, one of you must be wrong, shouldn't you want to make sure it isn't you?

Both also come up with how people deal with their own problems.

Too common to blindly spin up against blind spots/ego when solving problems.

Instead should:

teach brain to work in ways that don't come naturally
use compensating mechanisms (like programmed reminders)
rely on others who are strong in areas you are not

3.2 Practice radical open-mindedness

Radical open mindedness is motivated by genuine worry you might not be seeing choices optimally

Requires you replace attachment to always being right with joy of learning what's true.

A shift towards "higher level you" .

Ability to deal with things you don't know is more important than what you do know.

Decision making is two step process: take in info open-mindedly, then decide (see also section 5).

Don't worry about looking good/exposing your ignorance/weaknesses.

Remember looking for best answer; not best answer you can come up with yourself.

Distinguish between arguing and seeking to understand. Both have their places:

arguing for when people are peers
if one person is more knowledgeable more appropriate to have a student-teacher relationship

Believability

If you have a different view with someone who is more believable than you, you should make clear you're trying to understand their perspective.

If you're more believable, politely remind the other person of that.

3.3 Appreciate art of thoughtful disagreement

Goal of thoughtful disagreement is not to convince other party you are right, but to find out what's true and decide what to do about it

Both sides should be motivated by genuine fear of missing important perspectives.

Need to be reasonable, calm, collegial, respectful.

Most disagreements aren't threats, more opportunities for learning.

Two good tactics:

describe back to other person what they're saying
give others opp to talk for two straight minutes

All this is time consuming but worth it.

Important to prioritize what you spend time on and with who. There are lots of people to disagree with, unproductive to do so with all of them.

Explore ideas with most believable people you have access to.

Don't waste time disagreeing past point of diminishing returns.

Why doesn't thoughtful disagreement happen more? Because most people instinctively reluctant to disagree.

If two people are at a restaurant and one says "I like the food" the other is more likely to agree, even if not true.

This reluctance is lower level you's mistaken interpetation of disagreement as conflict.

That's why it's not easy: need to teach yourself art of having exchanges in ways that don't trigger reactions in yourself and others.

3.4 Triangulate your view with believable people who are willing to disagree

Smart people who can thoughtfully disagree are best teachers.

Also good to have two knowledgeable people thoughtfully disagree while you watch.

Also ok to turn over decision making when subjects are too complex for you to understand. You're giving into lower level self if you need to make your own decisions, even when not qualified.

3.5 Recognize signs of closed and open mindedness

Close minded people:

don't want their ideas challenged
are more likely to make statements vs asking questions
focus more on being understood than understanding others
say "I could be wrong", then follow it up with a statement, which is just perfunctory, way to convince self you're being open-minded
prevent others from speaking
have trouble holding two thoughts simultaneously
lack humility

Open minded people

are curious about why there is disagreement
ask questions, also consider relative believability

3.6 Understand how you can become radically open-minded

Regularly use pain as guide towards reflection.

Mental pain often from being too attached to an idea when person or event comes along to challenge it.

Make open-mindedness a habit. Try to consistently use anger/frustration as cues for being more thoughtful.

Know blind spots, helps to:

record circumstances in which you've consistently made bad decisions
ask others to help too
if you find yourself in an area like this again realize you're taking a huge risk

If a number of different believable people say you are doing something wrong and you are the only one who doesn't see it that way, assume you're biased

Meditate.

Be evidence based.

Realize at some point it's better to stop fighting for your view, and have faith in your decision-making process and accept what believable others think is best. Not doing this is dangerously arrogant.

More open-mindedness doesn't mean losing assertiveness. It should actually increase confidence because it increases odds of arriving at correct decision.

Are you up for it?

Are you willing to fight to find out what's true?

Do believe it's essential to your well being?

Do you need to find out if you're doing something wrong that is standing in the way of achieving your goals?

If no, accept that you'll never live up to your potential.

4. Understand People are Wired Differently

Most attributes are double-edged swords, can both help and hurt, depending on application.

Important to know how others and yourself are wired.

Desire for meaningful work and relationships is genetically programmed.

Subconscious

Large parts of our brains don't do logical things.

Your subconscious from the "two yous" thing from earlier isn't just animalistic and something that needs to be suppressed and overruled. It can also be source of creativity/good ideas. Many people believe best way to learn is to just cram more and more into your conscious mind and make it work harder, but often counter productive.

But don't want to just immediately act on subconscious things, should examine them with concious mind and look at them logically. Writing things down is helpful.

Again, the most constant struggle between thinking and feeling.

Amygdala (feelings, inner you, come quicker and in spurts, then subsides) vs prefrontal cortex (logical, outer you)

The biggest difference between people who guide their own personal evolution and achieve their goals and those who don't is the former reflect on their amygdala hijackings and put in context of what they really want.

Habits

Greatest challenge: having higher level you manage lower level you.

Best way of doing that: consciously develop habits that will make doing things good for you habitual.

Good habits get you to do what your "upper-level you" wants Bad controlled by lower level you, stand in way of upper.

Habit facts:

habit is most powerful tool in brain's toolbox
basal ganglia, not even concious of it
form in 18 months
three step loop (trigger, routine, reward)

In practice, -if you really want to change, best thing to do is choose which habits to acquire/get rid of, then go about doing that.

Dalio suggests write down 3 most harmful habits, pick one and commit to breaking it. Or acquire a good one if you want. Either way this will be very impactful to your life.

Dalio's best habit: using pain to trigger quality reflections.

Misc How People Are Wired Notes

Bridgewater uses briggs meyer, workplace personality inventory, team dimensions profile, stratified systems theory

Probably more interesting if you're building a team or hiring or something.

Dalio: having spent time with richest, most powerful people in world, beyond a basic level, there is no correlation between happiness and conventional markers of success.

A carpenter who derives deepest satisfaction from working with wood can easily have a life as good or better than president of US.

Having right people in right roles in support of goal is important.

Treat lower/subconscious you with kindness, don't nec need to fight with it all the time.

5. Learn How to Make Decisions Effectively

5.1 a Biggest threat to good decision making is harmful emotions

5.1 b Decision making is a two-step process (learning, then deciding)

Decision making processes more are more subconscious and complex than commonly understood.

Remind yourself it's never harmful to at least hear an opposing point of view.

Deciding is the process of choosing which knowledge you should draw upon.

Depends both particulars of this situation and broader knowledge. Weigh first order consequences against 2nd and 3rd order consequences

Never seize on first available option, no matter how good it seems, before you've asked questions and explored.

Dalio used to literally ask: am i learning? have i learned enough yet that it's time for deciding?

Learning comes down to two things:

Being able to synthesize accurately

converting a lot of data -> accurate picture
quality of synthesis determines quality of decision making

knowing how to navigate levels

5.2 Synthesize

Faced all the time with infinite number of things that come at you. Dalio calls them dots.

Need to be able to tell which dots are important and which aren't.

Some people just collect a ton of observations and opinions like lint.

One of most important decisions you can make is who you ask questions of:

make sure they're fully informed and believable
listening to random uniformed people is worse than having no answers

Everything seems bigger (more important) in the moment than it will in hindsight. For this reason, sometimes helpful to defer decisions.

New is overvalued relative to great.

Don't overfit limited numbers of dots.

5.3 Synthesize the situation through time

Keep in mind both rates of change and absolute levels; can be "getting better" while still being way below the bar.

Everything important in your life needs to be on a trajectory to be above the bar and headed towards excellent at appropriate pace.

Be imprecise. Understand what "by and large" means.

Remember 80/20 rule, understand what key 20% is.

Be an imperfectionist:

perfectionists spend too much time on little differences at the margin at the expense of important things
typically 5-10 important factors to consider when making a decision; important to understand these really well

5.4 Navigate levels effectively

Reality exists at different levels; each gives valuable perspectives.

Constantly seeing things at different levels and navigating between them.

> I want meaningful work that's full of learning.
>> I want to be a doctor.
>>> I need to go to medical school.
>>>> I need to get good grades in the sciences.
>>>>> I need to stay home tonight and study.

Pay attention to conversations, we tend to move between levels when we talk.

Use the terms "above the line" and "below the line" to establish which level a conversation is on.

When line of reasoning is jumbled, it's often because speaker has gotten caught up in below the line details without connecting them back to major points .

Only go below the line if it's necessary to illustrate something about a major points, and make sure you synthesize and tie things back.

Decisions need to be made at appropriate level and consistent across levels. Can't have 12 sausages and a beer for breakfast every morning and live a healthy life.

In other words: need to constantly connect and reconcile data you're gathering to get a complete picture.

Summary:

Remember that multiple levels for all subjects
Be aware on what level you're examining a given subject
Consciously navigate levels rather than see subjects as undifferentiated piles of facts that can be browsed randomly
Diagram flow of your thought processes using outline template (on page 250) shown previously

Do all of that with radical open mindedness.

5.5. Logic, reason, common sense are best tools for synthesizing reality

Another higher vs lower self principle.

Fundamentals of good decision making are relatively simple and timeless

"Until you make the subconscious concious, it'll direct your life and you will call it fate." - Carl Jung

5.6 Make decisions as expected value calculations

Raising your probability of being right is valuable no matter what your probability of being right already is.

Knowing when not to bet is as important as knowing what bets are worth making.

Best choices are ones that have more pros than cons, not the ones that don't have ANY cons.

5.7 Prioritize by weighing value of additional info vs cost of not deciding

Some decisions best made after acquiring more info, others immediately.

All of your "must-dos" must be above bar before you do your "like-to-dos".

Prob won't have time to deal with unimportant things, better than not having time to do with important thinks.

Don't mistake possibilities for probabilities.

5.8 Simplify and 5.9 Use principles for decision making shortcuts

Get rid of irrelevant details so essential things and relationships between them stand out.

Principles are a way to both simplify and improve decision making.

Realize almost all "cases at hand" are just "another one of those".

Figure out which it is, then apply well thought out principles for dealing with it.

Will allow you to massively reduce the number of decisions you have to make.

What Dalio does:

slow down thinking so you can note criteria you are using to make decision
write criteria down as principle
think about those criteria when you have an outcome to assess, and refine them over time

Identifying which "one of those" a situation is is like id'ing an animal.

Can be challenging since many since many cases are hybrids, in which case you have to weigh principles against each other.

5.10 Believability weight decision making

If disagreeing, see if you can start by agreeing which principles should be used to make a decision.

5.11 Convert principles into algorithms and have computer make decisions too

5.12 Be wary of AI without deep understanding

Sometimes can use to test how decision would have worked in past/other simulations.

Not super practical, prob more relevant for investment (and maybe fantasy football if you have a sweet model) decisions.

]]>

BP Oil Spill

Wed, 12 Aug 2020 00:00:00 GMT

BP Oil Spill

In April, 2010 the Deepwater Horizon drilling rig exploded off the cost of Louisiana, starting one of the largest environmental disasters in US history that ultimately leaked 4.9 million barrels of oil into the Gulf of Mexico.

In 2011, I finished grad school with degree in applied economics, and an interest and experience with environmental problems.

It's not that surprising that I soon started working for an environmental economics consulting company contracted to the government to (among other things) quantify the value of the lost recreation on Gulf beaches due to the spill.

Trips lost due to the spill

The lost recreation value due to the spill basically came down to:

lost value = n of trips forgone * value of each trip

I worked mostly on quantifying the number of forgone trips. In theory:

trips forgone = trips (no spill) - trips (actual)

Both numbers on the right hand side involve varying degrees of guessing.

Take the easier one: number of trips that actually took place. How would you go about calculating that?

"Ideally", maybe you have turnstiles at the entrance to every single beach in the gulf, then you can just count up the totals and see. In practice, obviously that's impossible

Instead we did some standard surveying and sampling techniques, dividing the gulf into segments and flying a plane up and down the coast multiple times a week snapping aerial photos. The photos were sent back to company headquarters, where we had a team that would count the people.

But snapshots only tell you the number of people at any particular moment in time. To go from that to full trips you need to get some sense of how long people were there.

For example, if everyone at the beach got there early and stayed all day, then the n of people in your picture == n of trips. Everyone was just there all day.

If everyone stayed for exactly (and randomly) half the day, then you'd have to multiply your snapshot count by 2. You counted the people there during the time you snapped a picture, but each person at the beach that day only had a .5 probability of being in the picture.

We got these durations (among other information) from a staff of 100+ people on the ground conducting interviews.

The result is cascading series of multiplying:

n people in a photo * 1/average length of time people stayed * 1/(how often you took a picture)

etc.

And that's just to get the actual number of trips taken post spill, which is the conceptually easier number. More abstract is the hypothetical: how many would people have taken had there been no spill?

Estimating the counterfactual

A few possible ways to approximate the number of trips people would have taken if there had been oil spill:

Figure out how many people took trips in 2009 (right before the spill).
Look at how many people took trips to places not affected by the spill, extrapolate based on that.
Figure out how many people took trips after things returned to normal.

All of these have issues. No one had the data for (1) — doing all the counting and interviewing for actual trips in the aftermath of the spill was very expensive, and no one had been doing anything like it pre spill.

(2) required good, comprehensive data on how the spatial impacts over time, which was another effort. Also it was possible more people than usual might be visiting non-affected sites, which would make the true number harder to calculate.

That left (3), which had its own issues (How do you know when things have returned to normal? What if things never did? What if people who hadn't been taking trips went all at once after things got back to normal, overstating the "baseline" n of trips lost?)

To deal with that, the government (specifically NOAA, who was in charge) decided we should continue counting people on the beach until the number of trips leveled off and stayed that way for a year.

To deal with seasonality inherent in recreation trips to the gulf, that required two years of sampling after things had returned to normal (a year longer than strictly necessary) in order to figure out what "normal" was.

Controlling for weather

Another issue with using later, "back to normal" years as a baseline is potential uncontrolled for differences in other factors that might effect trips.

For example, imagine the weather in 2010-2011 (when oil was affecting recreation trips) was unusually cold. Pretend it was freezing, too cold for anyone to go to the beach

What would have been the impact of the spill on recreation in that case? Nothing! If the weather was so bad and cold that people wouldn't have taken any trips anyway, then the spill didn't actually have any effect on recreation.

Obviously in real life, the weather wasn't that bad, but we had to account for issues (another one was differences in gas prices between years) like it.

Officially, we did so using some fairly basic statistical techniques (adjusting broadly for good and bad weather days across years), but — as a check — we also built a more complicated parametric model that gave us more fine grained control, I helped design and did all of the coding for that, which was fun.

Me and BP

When I got there in the fall of 2011, the data collection effort was well underway, but we hadn't started turning that raw data into counts of overall trips, much less trips lost.

My role became programming up all processing, multiplying and adding. I did it in Stata, which I had used a bit previously, but not that much (I haven't used it in since ~2013).

Since it was for litigation with a lot stake financially, we had a subcontractor doing the same weighting-up coding, but in SAS.

I have many (fond?) memories of talking through obscure data edge cases with him, trying to figure out why our trip totals (totaling in the 100s of millions) differed by 17 trips or whatever.

Aside from the unfortunate circumstances, I enjoyed working on the project. We had a team of big name academic experts, and it was fun hashing things out with some of the same people that had written my college textbooks and coding up their ideas.

Working on it was a big part in me realize I liked programming (and data and analysis), even more so than working specifically on environmental problems per se.

]]>

Quadratic Voting App

Wed, 05 Aug 2020 00:00:00 GMT

Vote Squared - Quadratic Voting App

Built April-May 2020, mostly during the kids naps while I was at home with them due to COVID-19.

www.votequared.com

Deciding in Groups

Voting is the obvious way to decide something as a group.

When most people think about voting, they probably envision "first past the post", where everyone votes for one of two or more options, and the highest vote-getter wins.

But with more than two options, it's not at all guaranteed that a majority of people will prefer the winning option.

One option to deal with this is ranked-choice voting, where voters rank options by order of preference and an algorithm uses that data to pick a winner. It's becoming popular in the US (which is good), and NYC is set to adopt it in 2021.

While ranked-choice voting is a big improvement over first past the post, quadratic voting does it one better by allowing people to vote not only on their rank order of preference (Amash, Biden, Trump) but also the intensity of that rank (super high, meh, none).

It also penalizes extreme, inflexible preferences by giving you less votes overall.

votesquared.com

My friends will occasionally vote on stuff (a good weekend to get together, new rule changes in our fantasy football league, etc).

In the past we've always done first past the post voting, which isn't ideal. I wanted to work on my web app skills and figured I might as well bring us into a more enlightened era of decisionmaking while I was at it.

The result is www.votequared.com. How it works:

Everyone gets a budget of $25.
The more you vote for an option, the more it counts against your budget. Your first vote for any option costs $1, the next $3 (so $4 total for the two votes), etc.
If you want to use all your votes (5) on one option it'll cost you your whole budget and you won't be able to vote for anything else.

Best Fruit

So say we're voting on best fruit. And have the options: apple, banana, pear, grape.

My first vote for apple costs $1, after which I have $24 left to use on banana, pear, grape or more votes for apple.

From there, I can spend another $1 on my first vote for one of the other fruits, or $3 for my second vote for apple. If I did that then a third vote for apple would cost an additional $5.

The total cost for n votes on any particular option ends up being n^2. With a budget of $25, that means the most you can cast at most 5 votes for any particular option.

And more votes for an option eat up more of your budget. The end result might be something like this:

apple   4 votes ($16 total)

banana  2 votes  ($4 total)

pear    2 votes  ($4 total)

grape   1 vote   ($1 total)

So while a ranked order ballet would reflect the order of my preferences (apple, banana/pear, grape), quadratic voting is better because it gets across how much more I really like apples vs the other options.

Technology

I built the site using Elm for the front end and a simple Graphql API (using ariadne, which I like a lot) for the backend.

I host both on my hobby $5 a month DigitalOcean droplet.

I send transaction emails using mailjet's free tier, which is OK, though I've had some issues with gmails spam filter so far.

]]>

Python vs R

Sat, 01 Aug 2020 00:00:00 GMT

Python vs R

One thing a lot of beginning programmers stress about is deciding which programming language to learn. Today the big choice in working with data comes to Python or R

Comparision

My Books teach you Python, along with libraries Pandas (manipulating data), BeautifulSoup (for scraping), seaborn (data visualization), and statsmodels and scikit learn (modeling and machine learning).

R is a statistical programming language and an alternative to Python. You can basically think of it as Pandas + a few other statistical libraries. It's a fine option. I started out in R, and am still moderately fluent. It's been around longer and has a solid MLB/football analytics following. Like Python, it's open source (e.g. free).

When it comes to data analysis, Python and R are a lot alike. Python probably has the edge in machine learning with scikit-learn, and Pandas is excellent (and getting even better all the time). R has the tidyverse, which a lot of people like.

For non-data applications Python is definitely better. People use Python to build their own APIs and websites, run their own servers, control their computers, robots, whatever.

That's probably why until about 5 years ago the R/Python split was about 50/50, but since then Python is the much more popular option (search for "R vs Python market share").

Programmers are more likely to move from R -> Python than the other way around.

Also interesting is this study of programmer migration patterns by programmer blogger apenwarr. He found that R users are much more likely to jump to learning Python next than the other way around.

As apenwarr explains, nodes in red below are "currently the most common 'terminal nodes' — where people stop because they can't find anything better".

Python 3 (which what we learn in LTCWFF) is a terminal node. R users (and Matlab, Fortran, etc) tend to migrate to it over time.

So if you're going to end up moving from R -> Python anyway (or, in my case SAS -> Stata -> R -> Python), it might make sense to save time and start directly with Python? If this sounds intriguing definitely check out the books:

https://fantasycoding.com

https://codebaseball.com

]]>

Learn to Code with Fantasy Football

Sat, 01 Aug 2020 00:00:00 GMT

Learn to Code with Fantasy Football

Learn to Code with Fantasy Football (LTCWFF) — available at fantasycoding.com — is a book I wrote mostly over the Summer of 2019.

It covers Python, Pandas (Python's main data analysis library), SQL, web scraping, public APIs, data visualization, summary stats and modeling, all applied to fantasy football.

Audience

The main target for the book are people who are:

into fantasy football
have done a bit of their own analysis in something like Excel
don't necessarily have much background in coding
are interested in learning more
unsure where to start

Which was basically me about 10 years ago.

The book has worked better for people who want to get into coding in an easy, digestible way as opposed to people who just want to win their leagues and are more indifferent to coding.

In other words, if your main objective is state of the art "who do I start" advice, it's much easier and more efficient to just buy a subscription to Fantasy Math (or 4 for 4 or whatever), then put in all the time and energy going through this book.

I think most people get that, and most of my readers have been people who are excited learning to code for themselves.

Market Research

I got the initial seed of an idea from a reddit post, which was about R and didn't end up going anywhere (though author did reach out to me after my book came out and was really nice about it).

I did one of my own posts on reddit to gauge interest (which there was a lot of) and start collecting emails.

Writing Process and Experience

Writing the book wasn't that difficult. I think that's more of a function of the topic — I've spent a lot of time thinking about fantasy football and a lot of time doing data science type tasks at my day job — than any natural skill as an author. I would guess future books on other topics might be tougher.

I also enjoy teaching and believe I'm pretty good at:

synthesizing/summarizing and finding themes/commonalities among large amounts of information
picking out a path to knowledge and introducing things at the right time
getting in the mind of the reader and finding and avoiding potentially confusing spots they might trip up on

At first I tried to write in prose form, but found that was way too slow, mostly because it was too hard to keep from trying to edit/rewrite/make perfect sentences as I went.

It was much easier for me to write out bullet points of the concepts I wanted to get across, even down to the sentence level (I do these writeups on my site the same way).

So I bulleted out the whole book, then rewrote it as prose.

Ray Dalio and Navigating Levels

I really like Ray Dalio's book Principles, and I often thought about the part in his book on navigating levels while writing and bullet pointing everything out.

Reality exists at different levels and each of them gives you different but valuable perspectives. ... We are constantly seeing things at different levels and navigating between them, whether we know it or not, whether we do it well or not... For example, you can navigate levels to move from your values to what you do to realize them on a day-to-day basis.

> I want meaningful work that's full of learning.
>> I want to be a doctor.
>>> I need to go to medical school.
>>>> I need to get good grades in the sciences.
>>>>> I need to stay home tonight and study.

According to Dalio, when a line of reason has gotten jumbled, it's often because the speaker is jumping around different levels without showing how they connect.

That's why, when conveying information, it's important to:

Be aware what level you're examining a given subject.
Consciously navigate levels rather than see subjects as undifferentiated piles of facts that can be browsed randomly.

Of course, I had always known that in theory, but explicitly keeping it in mind probably made the book better and has a lot to do with how the book has been received.

Format and Delivery

I released the book an en ebook (pdf, code, datasets, Anki cards) on Gumroad (update: I've moved to SendOwl), which I've generally been happy with.

It's really meant to be coded along with, and I think a pdf format is the best way to do that.

I've had a requests for hard copies, and in the future I may consider other options like Amazon publishing for those.

Reception

Reception has been very positive.

I offer a 30 day refund, and have had only a few people take me up on it.

As far as criticism (which I embrace — it's the most efficient way to make the book better by far), I've had a few people spot typos, which I expected given I never formally hired a copy editor. One benefit of the ebook format is I can fix them and rerelease it immediately, which I've done as they've come up.

Initially I didn't include any end of chapter exercises or problems, and I heard from a few people who wanted to be able to test themselves on the material. So recently I went back and added more than 100 problems with full solutions (most of them code).

Future Plans

I think there are people in other sports who would appreciate a similar book. I've released a baseball version and may do other sports depending on interest.

I also think the material may be good enough as a general purpose (non sports related) introduction to Python, Pandas and general data science practices. I've had a few readers who didn't know anything about football (much less fantasy football) read this book and get a lot out of it, which makes me think that may be a promising avenue. If I did that, I may explore working with a traditional publisher as well.

]]>

Forest vs Trees and Coding

Sat, 01 Aug 2020 00:00:00 GMT

Forest vs Trees and Learning to Code

Blogger and beginning programmer Jakob Greenfeld wrote recently about learning Ruby on Rails, a popular library (rails)/language (ruby) for building web apps

I've never used Ruby on Rails, but this part resonated with me:

"Since everyone keeps raving about it, I used [a popular tutorial]. While the tutorial was helpful, I felt as if something immensely important was missing from it.

"This approach works to give you a rough idea of how things work in Rails. But at the end of it, I had seen a hundred trees but couldn’t see the forest."

"There was just no systematic discussion of the most important concepts. Many puzzle pieces that seemed important were mentioned in passing. But how should I know as a beginner which of them are truly important?"

II

Jakob's post reminded me of concept of levels Ray Dalio talks about in Principles:

Reality exists at different levels and each of them gives you different but valuable perspectives. ... We are constantly seeing things at different levels and navigating between them, whether we know it or not, whether we do it well or not... For example, you can navigate levels to move from your values to what you do to realize them on a day-to-day basis.

> I want meaningful work that's full of learning.
>> I want to be a doctor.
>>> I need to go to medical school.
>>>> I need to get good grades in the sciences.
>>>>> I need to stay home tonight and study.

When a line of reason (or a programming tutorial) has gotten jumbled, it's often because the speaker is jumping around different levels without showing how they connect.

III

I thought about that constantly when writing my books.

At the highest level, data analysis is:

> Getting interesting or useful insights from data.

But that's pretty broad, so let's jump down a level. More specifically, data analysis is:

>> 1. Collecting Data

Examples: scraping data, connecting to a public API, downloading some ready made datasets.

>> 2. Storing Data

Once you have data, you have to put it somewhere. Maybe this is in spreadsheet files in a folder on your desktop, as tabs in an Excel file, or a database.

>> 3. Loading Data

Once it's stored, you need to be able to retrieve the parts you want. This is easy for spreadsheets, but if it's in a database then you need to know some SQL — pronounced "sequel" and short for Structured Query Language — to get it out.

>> 4. Manipulating Data

This step is getting your raw data in the right format for analysis. This is usually the biggest/most time consuming step by far, and is what most of the book is on.

Common tools for this step: Excel, R, Python, Stata, SPSS, Tableau, SQL, and Hadoop.

>> 5. Analyzing Data for Insights

The final model, summary stat or plot that takes you from formatted data to insight.

IV

Though we haven't covered any details, even this very zoomed out level is useful. For example, it wouldn't be unusual for a total beginner to wonder, "should I learn Python or SQL?" Knowing at a high level that SQL is primarily for getting data out of a database (vs scraping, sophisticated manipulation or analysis) makes the decision easier.

That's why Greenfeld's frusteration with his Rails tutorial ("At the end of it, I had seen a hundred trees but couldn’t see the forest.") hits home. From the intro of both books:

"Everything in [this book] falls into one of the five [data analysis] sections above. Throughout, I will tie back what you are learning to this section so you can keep sight of the big picture.

"This is the forest. If you ever find yourself banging your head against a tree — either confused or wondering why we're talking about something — refer back here and think about where it fits in."

You can check them out here: fantasycoding.com and codebaseball.com

]]>

Fantasy Math

Mon, 20 Jul 2020 00:00:00 GMT

Fantasy Math

Fantasy Math is my active site for weekly start-sit Fantasy Football advice. Subscribers enter in their matchup info and the players they're deciding between, and get back optimal player and the probability they win.

Many sites (Yahoo, ESPN, Fleaflicker) give team win probabilities, but — as far as I know — Fantasy Math is the only one modeling player projections as explicit probability distributions.

Projections as Probability Distributions

The fundamental problem in fantasy football isn't maximizing your expected points, but maximizing the probability you score more than whoever you're playing that week.

Projecting distributions for a player's score (as opposed to point estimates or rankings) helps in two ways. It lets you take into account:

Variance, how boom-or-bust a player is (1).
Correlation, the tendency of players' scores to move together(2).

Both can account for scenarios where starting the guy who's ranked higher (i.e. gets the most points on average) isn't necessarily the guy who maximizes the probability of winning.

I'm obviously biased, but because it takes correlations and player variance into account, I think Fantasy Math is the best start-sit advice you can get.

The Second Best "Who Do I Start Advice" You Can Get

It's easier to understand why Fantasy Math is the best start-sit advice you can get by considering the second best start-sit advice you can get.

Fantasy Pros Expert Consensus Rankings.

I have an economics background, and I've always been a big believer in efficient markets and the wisdom of crowds. I draft by selecting the biggest bargain according to ADP, and my investments are in index funds.

ECR is clearly the fantasy football equivalent of all that. And so if the goal is maximizing point totals, efficient markets would suggest — just as index funds beat stock picking — that's the best most of us (even experts) can expect to do.

Fantasy Math is essentially ECR with correlations and player variance added on top. Because the goal in fantasy is beat your opponent rather than maximizing your total points, this gives Fantasy Math a slight edge.

The Fantasy Math Edge

While real, I think the Fantasy Math edge is small.

The main reason is that optimal start-sit decisions are mainly driven by the scale parameter (i.e. start the guy whose distribution is furthest to the right), which traditional rankings and point estimates get at fine(3). Correlations and shape (aka variance aka boom or bust) are secondary.

I'm not even sure — compared to just using ECR or Boris Chen (also based off of ECR) — the impact on expected wins/share of league winnings is worth what I charge, just as a pure expected value calculation. Probably depends on your entry fee.

But some people won't rest until they've uncovered every possible edge, and Fantasy Math is for them.

How It Works

The are two parts to making the model: (1) fitting the distributions, and (2) figuring out the historical correlations between same and opposing QB, RB1, RB2, WR1-WR3, TE, K, DST.

I did (1) by fitting a Gamma distribution (both the scale and shape parameters) to historical Expert Consensus Rankings (via the weekly mean and standard deviation). For (2) it's just a giant correlation matrix.

Then I can use (2) to take generate random, uniformly distributed sets of numbers with the appropriate correlation. Then I feed those correlated 0-1 pairs through the Gamma distribution and end up with correlated simulations.

Then when someone puts in their matchup info on Fantasy Math, it queries all those simulations, figures out the percentage of time they win, and returns it.

Technology and Stack

The modeling is done using the Python data stack, and the API is in flask.

I did the first version of the front end in React using HP's Grommet framework. This worked OK, but doing this on the side, I wasn't constantly doing front end programming and didn't look at it much in the offseason.

When I went to pick it back up and wanted to make a few tweaks, I found that a bunch of the React API and frameworks I had been using were out of date. Around the same time I saw an article on Elm, and started playing round with that.

I liked it enough that — when I had hernia surgery and had to lay around for a week — I took the opportunity to recode the entire site in Elm, which is what it's been the past two season.

I like Elm a lot and might do a separate writeup on it at some point. I do sometimes wonder whether it's not boring enough though.

Challenges

Traction

Fantasy Math hasn't seen wide adoption and remains fairly unknown. I think there are a couple reasons:

The fantasy space is very noisy and competitive, with plenty of quality free material.
I'm a lot better at building models than I am marketing them (working on this).
It's a challenge to concisely and effectively convey how the model works and why it's good.
People are interested in buying WDIS fantasy advice for a few weeks out of the year, which makes it difficult to design and test marketing strategies.

Modeling

One of big drawbacks to Bayesian Fantasy Football was fixed shape parameter didn't allow for boom/bust guys. That's more flexible in Fantasy Math, but not to the extent I would like. This is one of the reasons I'm redoing the distribution fitting using more advanced techniques.

UX

I want the site to work better on mobile. Ideally people would be able to import roster and matchup information too.

Pricing

I'm not sure on pricing. While I'd love for it to be reasonably priced and widely adopted, that's not happening currently. I like running the site and have learned a lot doing it, but now that I'm out on my own I may need to focus on things that are profitable.

It's possible the only way that'll happen with Fantasy Math is if I jack up the price and focus on the much smaller set of users who recognize the value. They're out there (I've had people write saying they'd pay hundreds of dollars for it), and one of my goals this offseason is to figure out whether it makes sense to focus solely on them.

Footnotes

1 — Say you're down by 45 points going into a Monday night CLE-HOU game, would you rather start Will Fuller or Jarvis Landry? What if you were down 10 points?

2 — Imagine you have a close call — say Tyler Lockett vs Stefon Diggs — and your opponent is starting Russel Wilson. The fact Wilson and Lockett's points are positively correlated (about 0.44) might affect your optimal decision.

Precisely HOW it affects your decision depends on the rest of your matchup:

if you're heavily favored, the correlation might mean you should start Lockett as a hedge against Wilson blowing up
if you're the underdog, maybe you need Wilson to underperform AND Diggs do really well, and so Diggs maximizes your probably of winning

By modeling performance as correlated distributions, FM takes this into account. And not only this, but every pair of simultaneous correlations (e.g. the Lockett and Wilson are correlated, but they're also both correlated with Chris Carson and the QB Seattle is playing against) — between same and opposing QB, RB, WR, TE, K, DST.

3 — No matter what the correlations are, Fantasy Math is never going to tell you to bench Christian McCaffrey.

]]>

Bayesian Fantasy Football Writeup

Wed, 15 Jul 2020 00:00:00 GMT

Bayesian Fantasy Football

Starting in 2013, I sold weekly fantasy projections from a Bayesian model I built. The model used preseason draft rankings as priors, then incorporated weekly results for an updated posterior. I ran the site for four seasons before switching to an improved, non-Bayesian model and relaunching as Fantasy Math in 2017.

Motivation for site grew out of the (still held) belief that fantasy markets were mostly efficient, and that — with obvious exceptions for suspended guys and handcuffs — ADP probably did as well or better than anything else for WDIS decisions in week 1

But ADP is built on the best knowledge available preseason, and new information (mainly how guys actually performed) comes in weekly. I wanted a way to reconcile those prior beliefs with new data, and figured a Bayesian model was the natural way to do that.

What I did:

Got historical ADP data from myfantasyleague.com and fit it to a Gamma distribution, where ADP and ADP^2 entered linearly into the alpha (scale) parameter and beta (shape) was the same for all players in a given position and scoring system.
Wrote the Bayesian portion of the model in R with Jags, setting the prior so alpha would start at the ADP-fit model, then update as we added game results.

The model also controlled for home/away, strength of opponent, (via how many points they had given up to that position the previous 5 weeks), and (roughly) whether a player was a starter.

I didn't know anything but Stata at the time, which I used to fit the original, pre-season gamma as a function of ADP portion. I learned R in order to build the Bayesian piece.

In hindsight, and after revisiting probabilistic and Bayesian models again recently, I'm not I understood the math and techniques behind what I was doing that well, but I willed it into existence, and it worked OK.

Even now, I don't think anyone else is projecting performance distributions so in many ways it was a trailblazer.

Assumptions and Issues

The model basically assumed players had some fixed, hidden (but approximated by ADP) and unchanging distribution that we slowly uncovered via weekly draws over time.

That fit the bill for some (even a lot) of players, but not everyone, and the biggest problem with the model is it didn't do well on those players. It wasn't designed to handle large changes in players situation (a handcuff with new opportunity, a WR who's QB got injured, etc), and in that way probably would have worked better as a ancillary, supplemental model.

Another problem was the fixed beta parameter of the gamma meant rankings were monotonic and all dependent on the alpha parameter. This wasn't the end of the world (everyone else's rankings are like that too), but it meant missing out on some of the main benefits to projecting distributions, namely the ability to take into account the boom-bust ness of different players.

Technology

My front end knowledge was limited at the time, so Bayesian Fantasy Football was a Wordpress site with one plugin that took care of membership and another that served csv's as tables.

Every Monday night I'd kick off some code to grab the latest results (eventually I learned Python to do this it), ran everything through model, saved a new csv and uploaded it to the site.

As a teaser, I offered the top 5 guys at each position for free, but I'm not sure that was ever useful to anyone.

Bayesian Fantasy Football vs Matthew Berry (Moderated by Keith Olbermann)

To complement the paid, weekly in-season advice, I wrote up a free draft article, where I basically advocated drafting based on ADP, which I made sound more exciting by calling it drafting according to the wisdom of crowds (1).

In it, I included an analysis of — within any single position — how the difference between ADP and Matthew Berry (whose was well known and whose historical rankings happened to be online) was related to end of season points

For example, if Berry had Aaron Rodgers as the 5th best QB, and ADP had him 3, would that difference (5-3 = 2) be associated with more or less points (via a basic regression) at the end of the year? In other words, who was better, Berry or the crowd?

I did it over a couple years (I think three) and positions (four — QB, RB, WR, TE) and found for most of them the difference was statistically insignificant, but for the ones that weren't, the crowd beat Berry 4-1.

I got this article published on Lifehacker, whose sister site Deadspin had it up for a bit too. That night Berry was on Keith Olbermann's show on ESPN and Olbermann asked him about it, which was funny (Berry said he read it, but thought it was silly).

Bottom Line

I still like the idea of ADP as priors and weekly results as data and think the Bayesian Fantasy Football model was ahead of its time in a lot of ways. But it was restrictive enough that, when I developed a better approach (the current Fantasy Math model), it made sense to scrap it.

Footnotes

1 — This article was well- ("changed my perspective on drafting") recieved ("made my draft significantly less stressful... went from last place ... to first") — and I'd still generally advocate drafting this way.

]]>

EU Carbon Market Thesis Writeup

Wed, 08 Jul 2020 00:00:00 GMT

thesis

Draft Pick Trade Calculator Writeup — Nathan Braun

Wed, 01 Jan 2020 00:00:00 GMT

Draft Pick Trade Calculator

The Draft Pick Trade Calculator was a tool a built in 2015 in order to (1) help me with pre-draft pick trades and (2) learn React.

I still think the method behind it is theoretically sound and I might dust it off again someday, although there were a few issues I'd want to sort out.

Putting a Value on Each Pick

Pretend you're in an auction league — think of how much of an advantage would it be to start with $250 when everyone else has $200. All else equal, you'd likely be the favorite heading into the season, no?

Or consider a head-to-head match up between two teams, one with a starting line up of players who go (on average) for $137, the other $97. You'd expect the team with more value would win more often, right?

In a nutshell, this is how the pick calculator worked. Estimating the value of draft picks is easy, all you need to do is think about the value of the player you'll be able to pick there. Estimating the value of players is easy too, we have the perfect metric in Average Auction Values (AAV).

AAV is a great proxy for the value of each pick for the same reason that index funds are good investments. Efficient markets. Average Auction Values are prices. They have everything we need (upside, injury risk, scarcity at position, etc) baked in.

Valuing picks this way, and making trades that increase it, is a way to effectively start off your season with a higher budget than anyone else.

Translating Auction Values to Picks

But how do we actually translate AAV values to picks? On the surface it seems like we'd just want to sort AAV in descending order (e.g. CMC - $55, Saquon Barkley - $53, ...), slap those values on picks (1st OVR - $55, 2nd $53, ...) and call it a day.

This is close, but not exactly right.

Instead, we can rely on the fact that — in every individual league — SOME players will drop. We don't know who these will be ahead of time, but — as long as we commit to drafting value, whoever it is — we can plan on it and value our picks accordingly.

The question is then, on average, what AAV can we expect at each pick? To figure this out, I queried the MFL public API for results from more than 6000 auction and snake draft leagues to calculate what type of value we can expect on average at each pick.

Here are the results:

Note the different colored curves denote percentiles. The light blue line (P50) indicates the midpoint best-player AAV (normalized to a full 12 team, $200 budget league) you can expect available at any given pick. We can interpret these lines as varying levels of league competitiveness. For example, assuming every pick is very near (99 percentile) the best player available gives us a value chart equal to the dark blue line above.

League Competitiveness Plays a Role

One thing this chart makes clear is the expected value of a draft pick is based on how competitive your league is. Early picks are worth relatively more in competitive leagues.

This is intuitive: the tougher a league, the quicker good players go off the board, making later picks worth less than they’d be if no one knew what they were doing and bargains were prevalent.

I thought it was a reasonable assumption that that leagues where draft pick trading is prevalent are generally more competitive, so the calculator assigned picks based on the 90% percentile.

Caveats and Limitations

Following this calculator helps increase the auction value of your entire roster. I've noticed the calculator tends to be more in favor of trading down that most fantasy players, and I think this is why.

Having a valuable whole roster is a good thing overall, but it's possible it might lower your probability of winning — especially early in the season — by effectively recommending you swap high caliber starters for depth. Whether this helps you in the long depends on how you can turn that depth into starting lineup power.

In leagues without a lot of trading, it might be more appropriate to judge the impact in terms of auction amounts on your starting lineup. If I dust this off I might add in that feature.

I also think the MFL auction value data might be a bit thin. Higher quality (e.g. recent, with known scoring systems) average auction value data would make this tool more useful.

Tech

This was my first project in React. I think I also used Redux. If I were to revive it today I'd use Elm.

]]>