Alexander Etz: Understanding Bayes: How to become a Bayesian in eight easy steps

Alex Etz just wrote a really elegant blog post about his recent paper with Beth Baribault, Peter Edelsbrunner (@peter1328), Fabian Dablander (@fdabl), and Quentin Gronau in the special edition on Bayesian statistics for Psychonomic Bulletin and Review. In their paper they propose 8 papers that are worth reading to get into Bayesian statistics, giving a nice review / rating of these papers, as well as some others (that they still recommend, but maybe not as a crash course in Bayesian stats, but for those of us who want to learn more). If you’ve ever caught yourself thinking, I’d like to know more about Bayesian stats, but I don’t know where to start, then this paper is *literally* for you (and, well, for me, because I’ve been thinking this for too long…).

There was also recently a nice post by Wayne Folta about two packages in R that make Bayesian (regression) statistics a bit easier. Get it, Bayesians!


Colin Phillips: How to create a top journal by accepting (almost) everything

Colin Phillips has written a really interesting, and well thought out blog post about his experience as an editor of a special topic at Frontiers in Psychology (my most recent article was published in a different special topic (Models of Reference) in Frontiers – Talker-Specific Generalization of Pragmatic Inferences based on Under- and Over-Informative Prenominal Adjective Use). He found that while the model isn’t perfect, that it can be an improvement on the current model for publishing, discussing topics such as impact factor, time-to-publish, selectivity, etc. It’s a great read! Check it out here.


Preparing and Reading for Quals

Those of you who know me, know that I turned in my qualifying exam yesterday afternoon. Writing your qualifying exam in BCS is quite the affair, and it takes a surprising amount of preparation to get yourself through it. In many ways it’s actually a test of not just your knowledge in the field but also a test of perseverance. The following post is a list of things that I found helpful when prepping a reading list, and in reading for quals.

Making a list:

  • There are 3 sections to a reading list: general cognition, subfield (e.g., psycholinguistics, perception, cog neuro, etc), and then specific sub-subfield (e.g., pragmatics & acquisition, phonetics & categorization, etc). Generally the first section is broad and the shortest section, the list gets longer and more specific as you get more specific. For example (you can see what I read here):
    • General Cognition (17 papers): memory, attention, probabilistic models of language and reasoning
    • Psycholinguistics (20 papers): comprehension and productions
    • Pragmatics & Acquisition (58 papers): audience design, information structure, bayesian modelling, developmental pragmatics, implicature, communicative efficiency, etc
  • A good place to start might be to ask upper year students in a similar field to view their reading lists, this might help to figure out what kinds of papers you might want to have in the first two sections
    • Ask them if there were some papers that stood out as being helpful or unhelpful – having read them, they definitely know which papers were better than others
  • Consider which papers are considered to be classics in your field, papers that everyone should have read (e.g., Grice, 1975; Levinson, 2000)
  • It might also make sense to ask other professors who are specialists in your subfields if they have any recommendations
  • Your initial list will be super long – I think at one point my list of possible papers was 140+ papers
  • After building an initial list, you can ask your advisor(s) / committee members to look over the list and add / remove papers as they think reasonable
  • Finally, you should sit down with your advisor(s) and chat about the final set of papers of the list, this might be a good time to chat about what exact dates you will be writing your exam on

Reading for quals:

  • This is honestly the first stage of the exam that will test your perseverance: at this point you should know when your exam is, and how much time you have to read
  • Make a study plan: just reading at your own pace might not be the best strategy if you want to get through your list
    • If you set your exam far enough into the future, then you have the time to properly plan a schedule for getting through  your list
    • Previous lab mates of mine looked into the length of their papers and set up goals for how many days they thought it would to read each subsection
    • I used a website called strides (it allows you set goals and deadlines, and it will give you midpoint goals, and will predict how many papers you will likely get through by your deadline based on your progress to date).
  • Staying focused:
    • Pomodoro timer: Dave Kleinschmidt mentioned that he used one during his quals. The idea behind them is that you work for a short period of time (25 mins), then you get a short break (5 mins) – the nice thing here is that if you want to check your emails, or your facebook, waiting for 25 minutes is pretty easy to justify.
    • Forest:  While you can easily avoid checking things on your computer while reading, I found it much harder to not check notifications on my phone. So I really had to cut myself off of both. This app prevents you from checking your phone, by getting you to plant a digital tree. If you leave the app while the tree is growing (the time it takes for the tree to grow depends on you, but I recommend growing a tree for the same time as your pomodoro timer, or longer), your tree dies. If your tree grows to its full size you are rewarded with some coins that you can use to buy more trees for your forest (you can also label your trees in the forest with what you were focused on for that time). It’s weirdly motivating.
    • Consider printing out and reading your papers on paper – I later typed my comments into Mendeley, which helped me go over the things that I read.
  • Be organized: later you’re going to have to look up things from your papers and figure out which papers are relevant to what you are writing about.
    • Organize your papers: I used Mendeley, and kept all of my papers in one folder labelled “Quals”, this is helpful much later when building some reference lists.
    • Keep notes: I wrote both notes in the paper in-line, and then wrote a note summarizing the paper (this included my thoughts on the paper, things they discussed in the paper, models they considered, and points they argued for/agains).
    • Tag things: some people suggested keeping tags in Mendeley to be able to look up papers by keywords – I found this less helpful.
  • Reading strategies:
    • Originally, I tried reading from the start of the list to the end of the list – this strategy was fine for the first section, but it was hard to keep my interest this way, and also, I was reading over the holidays.
    • Instead, I wound up picking subsections to read – so, for example, I read all of the papers on Developmental Research in implicature and reference (12) across 3 consecutive days – this made sense, and luckily these papers we categorized correctly, so I could draw connections between them.
    • Try to keep some notes about things you are thinking about during your reading, maybe also think of some questions you hope to have answered by the time you finish your readings.
    • Know that you will need to take breaks, even days off.
    • Keep up with your regular activities – I found that still going to the gym, yoga, family plans, and plans with friends helped constrain my time (thus, reducing the time I had to procrastinate) and gave me something to look forward to.
    • If you are running out of time, knowing at the very least what the papers you might not get to fully read are broadly are about, is better than not even looking at them.

Writing Quals

I thought pretty carefully about how I wanted my quals to go and what I thought I needed to make it through. Akin to reading for quals, writing quals takes a lot of perseverance. I mean, you need to know a lot, need to have thought a lot about the greater picture behind your readings, but by the time you get to the start of your quals, you’ll probably have a good feel for what the greater picture is from your readings. For example, I posted the following just before my quals:

Anyways, here’s a short list of things that I did in preparation for quals:

  • Prepped some food: in the week leading up to quals I made a few meals that I knew I could freeze and defrost quickly for a meal
  • Stocked my kitchen with essentials: I had multiple kinds of breakfast foods, snacks, fruits, teas, coffee, chocolate(!!!), and picked up some ice cream mid-quals
  • Fixed my sleep schedule: I normally sleep weird hours, but I managed to get myself on a sleep @ 1, wake @ 8:30 schedule in the days leading up to my quals
  • Prepped for sound situations: sometimes I need to work in silence, sometimes I need music I can tune out, sometimes I need 90s R&B pump up jams – in prep, I made some playlists, stocked up on ear plugs, brought home my stereo headphones, and informed my neighbors of what I was up to (they offered to try to keep the noise to a minimum over the weekend, which was really, really nice of them)
  • Cleaned my house: I don’t think I could express the extent to which I appreciated my decision to do this in the hour leading up to the start of quals. My desk was completely empty, my living room / office was clutter free, my dishes washed and put away, my floors vacuumed, laundry neatly tucked away, and bathroom cleaned. This might not apply to everyone, but the lack of clutter really made it easier for me to not get distracted during my exam. I was careful to clean up my dishes in my first 5 minute break after a meal / coffee, and straightened the sheets on my bed every morning.
  • Went to yoga: while this isn’t for everyone, I started going to yoga in the winter of 2015 because I realized that I was really stressed out by work, and the hour I spent wondering how I could be so inflexible / focusing on my breath really helped me learn to relax. I felt pretty zen going into quals, and given how stressed I felt about it 2 hours earlier, and how well I slept that night, it was well worth it.

I recommend picking a time to start writing that works well for you. I chose to start my exam at 9:30 pm on a Thursday. When I told my advisors this, they both looked at me like I was crazy and mentioned that I must have had a good reason for this, which I did:

  1. I know for a fact that I have two good work times per day: in the morning right after my coffee, and after dinner until about 11
  2. I didn’t want to have a morning deadline – you get exactly 24 hours, and if I asked to start at like 10 am, that means that the last 8 hours of my writing time would be when I’d normally be sleeping or, worse, my least good working hours (I can work then, but my work is slow, and less good than any other time of the day)
  3. I knew I wanted to go to yoga before I started, because I knew it would relax me, and I love the yoga & meditation class at Tru Yoga on thursday nights

Writing quals itself wasn’t nearly as bad as I thought it would be. I got the questions at 9:30 pm, as I had requested, and was sent a set of questions from which I had to pick 6. I read through the list once, trying to think of which questions I knew I could answer, got to the end and (momentarily) panicked. It looked like a lot of work, and I really wasn’t sure that I felt certain enough in my knowledge to answer 6 of them. But fortunately I’m not one to be frozen with stress (I was once asked in an interview what I do when I get stressed, my answer was that I list the things I need to do, their priority, and then I deal with the smaller things – checking things off a list is a great motivator – until I get into a groove and the panic subsides. Fortunately, still true).

Another great choice in starting at 9:30 pm is that I picked 4/6 of my questions and immediately wrote out a single document outline for each topic. This outline document included the structure of my paper, and listed a bunch of citations and thoughts I had for the subsections. In some cases it even included an intro paragraph, and a lengthy bit of discussion for the end. Say what you will about late night working, but knowing it wouldn’t be the final draft, I felt free to write super informal thoughts. That night I went to bed feeling much better, and with a plan. Since I had 4 outlines, I figured that I could easily write 2-3 answers on my first full day (I was also using a browser countdown clock – 4 days is 96 hours, but I barely ever looked at it). That would then leave me with nearly 2 days to answer the two remaining questions. My first piece of advice for quals writing is set reasonable goals, and work to achieve them.

Before going to bed on the first night I actually wrote my submission email. I did this because there were some thoughts on my mind I wanted to write down (such as the fact that I was surprised in my own ability to focus and appreciation for the decisions I made leading up to my quals), but it also included some thoughts on how nice it felt to be supported by my lab mates, and the fact that I managed to have time to unwind (on Friday some lab mates swung by with some food and we went for ice cream, on Saturday between Q#3 and Q#4 I went to meet with lab mates for coffee, and started Sunday with a yoga class, and went for dinner with a friend). It also included a description of why I struggled to pick my last 2 questions: there were a few I had to pick from that I felt a little apprehensive about my knowledge, and then there were a few that I had to pick from that all posed different problems for me (one was easy but boring, one was interesting and I had posed myself but I didn’t think I had the answer and neither did the literature, and the last was the most challenging but the most interesting – in case you’re wondering, that’s the one I went with, and I couldn’t be happier). So, for a second piece of advice, start with what you know, and give yourself enough time to deal with the more difficult questions.

This slideshow requires JavaScript.

Perhaps a little known fact, but I track a not insignificant amount of information about myself. Above you can see my sleep patterns for the duration of quals. The best nights are definitely the night I started, and after I was done. Unrelatedly, I also realized a few days in that my strategy of backing up my work by running Time Machine was not giving me enough comfort, so I started regularly committing drafts of my answers to a private repository on git. For peace of mind, I recommend coming up with a strategy that works for you.

Quals are definitely hard, but they don’t have to be the worst. Sure, I was stressed during quals, but it wasn’t because I didn’t think I could do it. Keturah had told me beforehand that I would be surprised with how much I knew, and she was right, I knew or could reconstruct all the things I needed to make it through. The hardest part was convincing myself to keep writing (one of my least favourite things) for 4 straight days.

I tried to deal with this by taking reasonable breaks when I knew I couldn’t write any more (this included 3 hours I took to eat dinner and browse the internet one day), on the third full day I recognized I was far enough ahead that I could afford to go to yoga to help me unwind, and because I’m extremely extroverted I took some time to see friends. So, the third piece of advice is, you don’t need to write non-stop for 4 days, recognize that you need breaks and take them. 

Things went as I planned. I finished 4 questions by the end of the second full day. I decided which question I would work on in the morning, and told myself that I needed to pick a final question the next day. I did. The last question took me the longest, in part because it was challenging and in part because I was so close to finishing, I just couldn’t take it any more. I had to restart it 3 times before I finally got into the swing of things, and just free wrote ideas (see: Sunday night in my sleep charts). I woke up Monday and finished it with my late-night notes. I really appreciated writing that question, though, as I felt like it tied together many things that I started to see come together throughout my quals. It was interesting and I had never really thought of my work in the greater picture of things so much. It really made me want to engage more in other areas of cognitive science. While it wasn’t the greatest of times, and I’ll agree with Dave, I don’t think I’ve ever been so crazy as in the month leading up to it, or in the days I was writing (I apologize if I was terrible company in that time), but it was definitely eye-opening and I’m glad I did it.

If I were to offer one last, super important piece of advice, it’s really to know yourself and know your limits. It sounds cheesy, but I’ve always been a bit introspective, and as previously mentioned I track a lot of information about myself, and I think it was to my benefit to know what would and would not work for me going into the exam. I knew how much coffee or tea I needed and when to stop, I knew I needed to take breaks and that I couldn’t keep listening to the same music, I knew I needed to get out of the house and that I needed the yoga to unwind, and I knew when I was done that I had done some good work. Here’s hoping my committee feels the same.


Feeling Supported

I remember when I interviewed here that the thing that stood out to me the most was that all of the students talked fondly about how supported they felt during their quals. They all told stories about how their lab mates banded together and made them food to help them get through. At the time it sounded also slightly horrifying, given they talked about them as if you don’t have time to make food (that’s not the case, but if you are really into what you are doing, you maybe don’t have the mental energy to expend to planning a meal).

I was fortunate to have some meals prepped by my lab mates Zach, Linda, and Thaìs and I really, really, super appreciated it. I thought I had things planned out and that I would easily make it through with the food that I had, but it was nice to not have to worry about it, and also to know that they had thought about me.

It was also nice to have spent some time with Dave, Dan, and Göker over the weekend. They all put up with me chatty sort of crazily about what I was working on.

And finally, it was nice to have friends check in on me, whether it was on gchat (thanks, Esteban & Emanuela), twitter (thanks, Dave, Whitney, & Andrew), or facebook (thanks, mom).  I really did feel well supported.