learnmap

Table of Contents

Self-Directed Reading in Computational Cognitive Science

I am in a somewhat unusual situation employment-wise right now - you could think of my current role as an "applied grad student" of sorts. This is (I think) particularly well-suited to my personality, but jumping back into this sort of half-academia world is inspiring a good amount of motivation (read: terror) about the body of information I need to be learning at the moment.

Ergo, I think it is a good idea to actively keep track of the things I need to read, since the queue is constantly growing and keeping it all bottled in my head stresses me out. Books you read reduce your list of books to read by one book, but typically lead you to one-or-more books in return, and your priority list shifts accordingly whenever you consume any resource in this way.

Goals

The nice thing about my current position is that I have a good amount of control over where I can direct my learning - as long as I am useful enough in front of a keyboard, I am free to optimize my research interest in whatever direction I feel is most interesting / most lucrative.

At the time of writing (2019/06/25) my broad, ordered list of priorities is as follows:

  1. Become good enough at machine learning / artificial intelligence / etc to be good at my job
  2. Direct studies towards computational cognitive science, aiming to contribute to research in artificial intelligence through that lens
  3. Become good enough at math to deeply understand AI

I care deeply about these three things, in this order. The three of them are connected, so they should all be done relatively concurrently with a sort of weighted time/effort/attention heuristic applied to it. Likewise, there are a plethora of resources which probably could be well captured by multiple categories, so I'll have to use some judgment calls for organizational purposes (since org documents are hierarchical).

How To Use This Document

This document is primarily intended to be used as an emacs org-mode document, mostly just so you can collapse the bullet points down easily. I expect this document to get pretty long, and it will be fairly difficult to manage to order them in a way that appropriately conveys the intellectual dependencies required to read things effectively. I will try to order things within subcategories in a top-to-bottom fashion within each subsection (such that the harder things to read are lower along the list) and make annotations when that is not possible, but if you're curious about the order I actually read things then I encourage you to check my links page, and start from around February 2019 (you'll have to wade through my other links, though).

Generally speaking, if you see something with [ ] next to it, I have not completed it. If you see something with [X] next to it, I have completed it to my own satisfaction. If you see something which is listed in the syllabus which has neither of these things next to it, it is probably a paper which I read in something which does have one of those two things, most likely a reading list which may or may not be in the "Archive" section now.

Philosophically, I am trying to keep this a pretty non-easily-outmodable syllabus. That is to say, I've tried to avoid including papers which I believe will become outdated once they are replaced by some new state of the art. There are some exceptions to this (i.e. some of the Deep Learning stuff which is too practical to omit) but the idea here is I'd like this document to be useful to people aside from myself, and it would be sad if everything I write down here just becomes worthless to anybody but me at that time.

It should go without saying, but if a resource includes non-optional readings or exercises, do them unless told otherwise - I can't be bothered to reprint all of them here.

Current Priorities

  • deeplearningbook.org
  • All Of Statistics
  • BJS Grad Student Readings - Statistics
  • Elements of Statistical Learning

Syllabus

Artificial Intelligence

My goals with learning Artificial Intelligence are a little bit less academic than the other categories in this document; I am currently more interested in getting the bat into my hands, so to speak, to use AI as a tool with which I can solve problems I am confronted with. Learning these things in a rigorous, disciplined manner is still important, but I lean less theoretical with this point on my self-imposed learning triangle compared to cogsci and math.

Machine Learning and Artificial Intelligence

  • [ ] Pattern Recognition and Machine Learning (Bishop)
  • [ ] Artificial Intelligence: A Modern Approach (Russell and Norvig)

Deep Learning

Reinforcement Learning

Smaller Fields

  • [X] Active Learning (Settles 2012)

Cognitive Science

This category is a sort of funny one since "Cognitive Science" is not completely entirely it's own discipline, so it's hard to know what explicitly belongs in this category (as opposed to, like, neuroscience or AI). For simplicity, this point in the hierarchy has my resources which either A: explicitly bridge gaps between multiple fields, or B: have more to do with the practice of science compared to a math / ml resource. Likewise, there is plenty in cognitive science that I want to read that doesn't fit well within the scope of this document (for example, Psychology Applied to Modern Life) just out of being somewhat irrelevant to the "computational" part I'm going for, so this list is intended to be somewhat more directed.

Likewise, for being a relatively young (and fragmented) field, I think an unusually important part of being a student in cognitive science is actually reading papers of researchers who interest you. It's not that you don't do this as a graduate-level student in any discipline, but for cognitive science in general you don't have that many options other than that (compare to math, where you can just read any number of textbooks for the hundreds of years of progress in mathematics).

As this document evolves I expect this section to grow the most; currently for brevity's sake I have reading lists as the primary object in this section, but as I read things I think I will expand this to include actual papers that I think are "timeless enough" to serve as a useful entry for this syllabus, especially as I split this out into sub-fields.

Books and Textbooks

Computational Cognitive Science
Scientific Practice
  • [ ] Kuhn, T. S. (1970). The structure of scientific revolutions

Reading Lists

Course Syllabi w/ lists of readings

  • [ ] MIT The Brain and Cognitive Science one and two
  • [ ] MIT Computational Cognitive Science readings
  • [ ] MIT Probability and Causality in Human Cognition readings
  • [ ] MIT Statisical Learning Theory and Applications readings
  • [ ] MIT The Development of Object and Face Recognition readings

Specific People I am Interested In

  • [ ] Josh Tenebaum
  • [ ] Noah Goodman

Papers

Scientific Methodology
  • Strong Inference - Platt 1964
  • Fuzzy Methodology - Cohen 1992
  • Why most published research findings are false - Ioannidis 2005
  • The head and the hands - Koenderink 2002
  • Cigarette smoking: an underused tool in high-performance endurance training - Myers 2010

Math

Math learning resources are a bit more of an enigma to me because the amount of reading is so vast but comparatively more organized compared to other fields. Part of the fact that this is a self-instruction syllabus is that I just have no clue how good any of these resources are, despite there being so many. The below is probably enough to keep me busy for multiple years, even if I go very fast, and it definitely the least immediately clear to me how useful any of this will be (especially given that my colleagues are so skilled at math).

That said, I think it's healthy to always be learning math anyways, so I am not particularly concerned if some of this is not super immediately applicable to my career. I expect a number of these to be slanted towards refreshing my knowledge of things I would claim to know (i.e. linear algebra) but strong mathematical fundamentals are important to have for artificial intelligence.

More Basic

  • [ ] The Art of Problem Solving Volumes I and II (Perhaps an embarrassing inclusion? But perhaps not)
  • [ ] How To Prove It (Velleman)
  • [ ] Strang linear algebra
  • [ ] Apostol volume II
  • [ ] Principles of mathematical analysis (Rudin)

Statistics

Textbooks

Logic

  • [ ] Computability and Logic (Boolos)
  • [ ] The Logic of Provability (also Boolos) (only ch 1-5? 1-9?)
  • [ ] A Mathematical Introduction to Logic (Enderton)

Applied Math

  • [ ] Analysis on Manifolds (Munkres)
  • [ ] Mathematics for Computer Science (Lehman)
  • [ ] Lambda-Calculus and Combinators (Hindley)
  • [ ] Basic Category Theory for Computer Scientists (Pierce)
  • [ ] Topology (Munkres)
  • [ ] Introduction to the Theory of Computation (Sipser)
  • [ ] Principles of Model Checking (Baier)

Reference

  • [ ] The Princeton Companion of Mathematics

Other Resources

This part of the document I think is a lot more backwards-view compared to the "syllabus" section, but there are a number of things that I consider genuinely useful resources despite their relatively informal or unusual formats (i.e. not textbooks, papers, or lectures).

Videos

  • 3Blue1Brown's youtube channel is genuinely one of the best resources I've ever seen for developing visual intuition for mathematical concepts. In particular his sequences on Machine Learning, Linear Algebra, and Calculus are very watchable and would be starting points for any student of any of these fields.

Books

  • Godel, Escher, Bach: An Eternal Golden Braid (honestly required reading for anybody interested in cognitive science at all, level of computational rigor aside)

Archive

I will place things in here which I do not think belong in the main syllabus anymore, namely completed reading lists which I have pulled out the more relevant papers from for somebody interested in the intersection between AI and Cognitive Science.

Back to Top