I think I disagree (or at least, I'm confused by your model of passive grantmaking vs field strategists). [edited, I had misread] My understanding is that you think that a field strategist role is a very different role to a passive grantmaking role. Based on how you describe the field strategist role, they seems quite similar to me -- passive grantmakers are also in the business of developing high-level strategies for the field, and thinking about what needs to be happening/what's on the critical path. (And I think grantmaking is a pretty good way of getting feedback on field strategy, which has pretty bad feedback loops.)
FWIW my experience at CG is that we either have, or are developing, answers for Qs like: "what is our theory of victory for AI safety? Given this theory of victory, what is on the critical path? What projects and organizations on the critical path do we need to initialize?"
Having said all this, I would be excited if more people spent more time developing strategic taste (I'm working on a blog post on this!)
Hi Catherine, thanks for commenting + for our lovely chat last week!
> passive grantmakers are also in the business of developing high-level strategies for the field, and thinking about what needs to be happening/what's on the critical path.
Ah interesting, I didn't know this -- previously (in ~March of this year), I chatted with someone at CG and asked them about what high level strategy guides CG's grantmaking, and was told that there isn't really a high level strategy that incoming applications are evaluated on the basis of. I'm glad to hear that this is changing / has changed / that maybe the person I chatted with was wrong!
Looking forward to seeing your tips on strategic taste :)
Also, just to note as an aside, I don't mean to come off as against CG in any way in this piece -- just wanted to write up some thoughts on grantmaking in general, and used CG as one of the examples given it's the top funder of the field (but long wait time for grant evaluations, for example, is a problem at Longview and other major funders as well). I think CG has had an overwhelmingly positive impact on the field! (Wouldn't currently be at MATS without CG ofc! <3)
Yeah, that makes sense! Given I haven't got much information, it's hard to say whether it's different between teams, or between grantmakers, or has changed recently, or if there's something lost in translation between you and the person you were talking to, or between your piece and my interpretation of it, etc etc. (These kinds of telephone-y situations are always kind of hard to figure out imo.)
FWIW, I think you should feel extremely empowered to criticise CG! Outside criticism is great and valuable! :))
Catherine, I didn’t read Sophie as saying grant makers need to be strong executors? But Sophie can clarify I suppose.
> Publish detailed, pre-scoped project plans that applicants apply to execute.
* I think a cost of this is that often people will execute on projects better if it is their idea from start to finish and they feel especially passionate about just that project, compared to if we hand them a project someone else came up with. That said, AIM seems to work very well with your model of pairing founders with ideas someone else came up with, so maybe I am wrong about that.
> Hire full-time field strategists.
* I think this could make sense! I expect that it will often make sense for one person to develop deep expertise in an area, and develop a field strategy, but then they should probably also do passive and active grantmaking on that area, as even if you write out a really long and detailed field strategy it might be hard to convey everything, and having deep world models in your area is probably still very useful for grantmaking, rather than just finding a fresh grantmaker to read the strategy doc and go off and make grants.
Overall I’d be interested to see more of people coming up with ideas they want to see done, Launch Sequence style. Forethought does a bit of this, e.g. here.
I have been doing some ‘boosting middle powers’ grantmaking lately, and plan to also publish some of my field strategy thinking, perhaps as a reading list + open problems that ERA/Pivotal/etc fellows or others can attack.
> I think a cost of this is that often people will execute on projects better if it is their idea from start to finish and they feel especially passionate about just that project, compared to if we hand them a project someone else came up with.
Ahhh hmm, this makes sense -- though I think this effect might be person-dependent, or perhaps project-dependent? A couple thoughts come to mind:
- I might be typical-mind-fallacy-ing, but when I come up with a project idea, I'm often a bit worried that I might not have enough strategic taste compared to people who have been in the field for longer to be able to tell whether or not the project genuinely seems like a valuable idea. A lot of my research directions have come from / been heavily informed by discussions with others in the field whose judgment I trust, and this has made me a lot more comfortable + confident with pursuing them, which leads to better output!
(I suppose the above might only be the case for relatively new people to the field, though, and wonder if it might change for me at some future point or is already changing somewhat.)
- I think this might also be a case of whether or not you can 'project-pill' the person in charge of execution strongly enough for them to say, "this is an important unsolved thing / unaddressed gap that needs to be worked on, I *really* care about this problem and want to work on it" and for them to become passionate about the project as a result, even if it wasn't initially their idea.
Curious to hear more about your middle powers grantmaking, and excited to see your field strategic thinking!
Weakly held but I think one problem with the current AI safety grantmaking ecosystem is a lack of intellectual diversity. Everyone is reading ~the same blog posts and field strategy docs, hanging out with the same people, and a lot of discourse happening inside orgs as opposed to outside them. This creates consensus opinions that leads to our bets being correlated in ways that I think is dangerous in such a nascent and epistemically treacherous field as AI safety.
I strongly agree! One way to raise this diversity in thinking rapidly is by diversifying geographically -- use the money to found attractive regional AI safety hubs around the globe.
Exactly! The Future of Life Foundation (FLF, not to be confused with FLI) has been running with this same division of specialisation for two years now. We're big fans of Evan's similar approach. FLF is doing something like field strategy and currently hiring for project development leads: https://flf.org/jobs/
Thanks for writing this!
I think I disagree (or at least, I'm confused by your model of passive grantmaking vs field strategists). [edited, I had misread] My understanding is that you think that a field strategist role is a very different role to a passive grantmaking role. Based on how you describe the field strategist role, they seems quite similar to me -- passive grantmakers are also in the business of developing high-level strategies for the field, and thinking about what needs to be happening/what's on the critical path. (And I think grantmaking is a pretty good way of getting feedback on field strategy, which has pretty bad feedback loops.)
FWIW my experience at CG is that we either have, or are developing, answers for Qs like: "what is our theory of victory for AI safety? Given this theory of victory, what is on the critical path? What projects and organizations on the critical path do we need to initialize?"
Having said all this, I would be excited if more people spent more time developing strategic taste (I'm working on a blog post on this!)
Hi Catherine, thanks for commenting + for our lovely chat last week!
> passive grantmakers are also in the business of developing high-level strategies for the field, and thinking about what needs to be happening/what's on the critical path.
Ah interesting, I didn't know this -- previously (in ~March of this year), I chatted with someone at CG and asked them about what high level strategy guides CG's grantmaking, and was told that there isn't really a high level strategy that incoming applications are evaluated on the basis of. I'm glad to hear that this is changing / has changed / that maybe the person I chatted with was wrong!
Looking forward to seeing your tips on strategic taste :)
Also, just to note as an aside, I don't mean to come off as against CG in any way in this piece -- just wanted to write up some thoughts on grantmaking in general, and used CG as one of the examples given it's the top funder of the field (but long wait time for grant evaluations, for example, is a problem at Longview and other major funders as well). I think CG has had an overwhelmingly positive impact on the field! (Wouldn't currently be at MATS without CG ofc! <3)
Yeah, that makes sense! Given I haven't got much information, it's hard to say whether it's different between teams, or between grantmakers, or has changed recently, or if there's something lost in translation between you and the person you were talking to, or between your piece and my interpretation of it, etc etc. (These kinds of telephone-y situations are always kind of hard to figure out imo.)
FWIW, I think you should feel extremely empowered to criticise CG! Outside criticism is great and valuable! :))
Thanks for the post Sophie!
Catherine, I didn’t read Sophie as saying grant makers need to be strong executors? But Sophie can clarify I suppose.
> Publish detailed, pre-scoped project plans that applicants apply to execute.
* I think a cost of this is that often people will execute on projects better if it is their idea from start to finish and they feel especially passionate about just that project, compared to if we hand them a project someone else came up with. That said, AIM seems to work very well with your model of pairing founders with ideas someone else came up with, so maybe I am wrong about that.
> Hire full-time field strategists.
* I think this could make sense! I expect that it will often make sense for one person to develop deep expertise in an area, and develop a field strategy, but then they should probably also do passive and active grantmaking on that area, as even if you write out a really long and detailed field strategy it might be hard to convey everything, and having deep world models in your area is probably still very useful for grantmaking, rather than just finding a fresh grantmaker to read the strategy doc and go off and make grants.
Overall I’d be interested to see more of people coming up with ideas they want to see done, Launch Sequence style. Forethought does a bit of this, e.g. here.
I have been doing some ‘boosting middle powers’ grantmaking lately, and plan to also publish some of my field strategy thinking, perhaps as a reading list + open problems that ERA/Pivotal/etc fellows or others can attack.
Hi Oscar! Thank you for commenting :)
> I think a cost of this is that often people will execute on projects better if it is their idea from start to finish and they feel especially passionate about just that project, compared to if we hand them a project someone else came up with.
Ahhh hmm, this makes sense -- though I think this effect might be person-dependent, or perhaps project-dependent? A couple thoughts come to mind:
- I might be typical-mind-fallacy-ing, but when I come up with a project idea, I'm often a bit worried that I might not have enough strategic taste compared to people who have been in the field for longer to be able to tell whether or not the project genuinely seems like a valuable idea. A lot of my research directions have come from / been heavily informed by discussions with others in the field whose judgment I trust, and this has made me a lot more comfortable + confident with pursuing them, which leads to better output!
(I suppose the above might only be the case for relatively new people to the field, though, and wonder if it might change for me at some future point or is already changing somewhat.)
- I think this might also be a case of whether or not you can 'project-pill' the person in charge of execution strongly enough for them to say, "this is an important unsolved thing / unaddressed gap that needs to be worked on, I *really* care about this problem and want to work on it" and for them to become passionate about the project as a result, even if it wasn't initially their idea.
Curious to hear more about your middle powers grantmaking, and excited to see your field strategic thinking!
Thanks Oscar! I had misread Sophie on executing 🫠 edited
Weakly held but I think one problem with the current AI safety grantmaking ecosystem is a lack of intellectual diversity. Everyone is reading ~the same blog posts and field strategy docs, hanging out with the same people, and a lot of discourse happening inside orgs as opposed to outside them. This creates consensus opinions that leads to our bets being correlated in ways that I think is dangerous in such a nascent and epistemically treacherous field as AI safety.
Had a chat with someone at the ERA fellowship who called this "worldview decorrelation" -- have written about this here! https://thecounterfactual.substack.com/p/the-anthropic-ipo-is-coming-we-arent (see "3b. Independent grantmakers with different worldviews")
I strongly agree! One way to raise this diversity in thinking rapidly is by diversifying geographically -- use the money to found attractive regional AI safety hubs around the globe.
Exactly! The Future of Life Foundation (FLF, not to be confused with FLI) has been running with this same division of specialisation for two years now. We're big fans of Evan's similar approach. FLF is doing something like field strategy and currently hiring for project development leads: https://flf.org/jobs/
Also compare an outfit like Forethought (and likely others) which are more expressly focused on the upstream macrostrategy. FLF collaborates with Forethought, for example on our design sketches for a more sensible world: https://www.forethought.org/research/design-sketches-for-a-more-sensible-world