Facilitator: Spencer
Notes: Svenja
Agenda
Proposals Ideas in Advance
Notes
Open questions
Outcome: Proposal will be further refined in a process of crowdsourcing (slack posts)
All
Data Club/Non-Data Club
Fred/Spencer: make DC weekly again. Make it possible to make presentations shorter and have many, and increase the probability that it’s actual work in progress
Philip: Counter proposal - make it less data heavy (non-data club)
*
TODO
Facilitator: Spencer
Notes: Svenja
Agenda
Notes
Working Group organization, progress, plans at SWC (Tom)
More than pairwise collaboration between labs, project is intended to involve many labs and Gatsby and affiliated labs - harness the collective expertise
_
Idea for the project came about by brainstorming between the group leaders:
How do animals make decisions in complex environments, especially in foraging where there are competing interests? Two sets of questions: behavioural level (what strategies do animals employ, how does it depend on context…) and at the neural level (how do neurons drive this behaviour? Where are the regions influencing these decisions, and how does that lead to the spatial outcome of going to place x?)_
_
Structurally, there is a ‘Steering Group’ of PIs. Now, subgroups will be formed to do specific tasks (building arenas, behaviour, data analysis…)
Decision making is similar to here by consensual decision-making (proposals etc.)_
_Soon, everyone will be allowed to join the working group. Some of these people will be PhD students or PostDocs affiliated to a specific lab, but also people can be hired directly to the working group.
_
How would publications work?
Everyone should be on the paper, but there can be task forces e.g. 4 people just focussing on describing the basic behaviour - these are the people who took the paper from data to actual completion. General papers can have many contributors, with a table outlining who contributed what.
DeepMind
Highest level: 2D structure of teams and topics which interact and overlap.
3 elements to the organization: efforts (6 month long cycles, ideas can come from the top or from the bottom, within these there will be milestones etc.), projects (month/week-long cycles), every Wednesday there is a company-wide meeting - mostly research-updates (5 -10 min) from each team, the teams decide on the Friday before who presents (trickle-up of the most interesting thing)
Every team has their own project management tools, but it’s all transparent and available
_Frequency of meetings is far higher at DeepMind: Public googlecals allow to ping anyone for a 20min meeting to ask for their specific expertise. _
Advantages: more interactions, the asking person has to phrase their problem, the limited time frame allows for efficiency but more lateral sharing of information; This may organically lead into collaborations.
_There is meeting-free Thursdays. Every week you have meetings with a smaller group of people who do the same stuff as you, every two weeks with larger groups that will connect you with people who aren’t doing the same stuff.
Something DeepMind has we don’t need: Project Manager, since we have a small enough building to know everyone. _
_Suggestion (Tom): Expert groups on slack _
Facilitator: Spencer
Notes / Shadow Facilitator: Svenja
Agenda
Decide on repeated meeting time and frequency, choose facilitator + notetaker for next meeting (5mins)
Total time = 65mins
Notes
Summary Big Science, Team Science, Open Science paper (5mins)
Big science: e.g. Allen Institute - Mouse Brain Atlas – projects that are not accomplishable by one lab alone
Team science: larger than one PI-one lab groups – sociological implications: identify common goals and align to the goals
Questions: how should individuals contribute? How to reward individual creativity, discipline, etc.?
Only works if the bigger goal of serving a larger purpose is reinforced, and if decision making is transparent
Example of a field in which team science is further ahead: physics,astronomy
Open science: aims to share data rather than polished results, requires every researcher to have the tools to make data accessible
Rallying around principles, figuring out what is rewarding to people, what are people working towards, how do the goals of all stakeholders (funders, leadership, researchers) align, Idea that new tools can drive new approaches to science
General values to rally around is nice, but rallying around a project is easier
Current Researcher Journeys at SWC
PhD: time-limited, purpose: get a degree, but still can encompass very different motivations
Alternative paths – industry, staff positions (may lead to more collaborative work)
Post-Docs: leading up to bottleneck bc there’s more PostDocs than PIs, although that is becoming less hierarchical
Classic and alternative programs \
What about people (at any career level) who are happy to be involved in many projects and not being first author on them? Do we want to increase the importance of these, and if so how? How can this be funded if you are not the SWC/Crick?
Staff scientists: sustainable expertise, provide continuity
How to organize projects outside classical lab structure? (Admins, funding)
Project leaders rather than group leaders (See DeepMind notes)
It’s not just what people choose to do, but also what options they have. How can we prevent valuable talent from leaving due to lack of attractive options?
Look at things as a resource problem – resources tend to flow out of the building and expertise leaves the building constantly. How can we work to hold onto this investment of energy?
Tom mentioned that a big science project is being assembled at the SWC. This project is related to decision making in the mice.
Dichotomy between middle author staff scientist track vs first author lonely wolf PI track:
It shouldn’t be this binary
Option of not having this first/middle author distinction on papers
_Funders want staff scientists to be associated to grants, but that is a debate _
Brainstorm Utopian Vision (15mins)
If you want to keep talent, you need to provide incentive
_
Maybe you have staff scientists and time-limited contracts at the same time? Maybe the problem is not that we need to keep people longer, but giving the people the possibilities to accomplish projects efficiently?
Example DeepMind: give people other reasons than high-paying contracts to want to stay - people are happy to stay bc they want to be part of it_
_
E.g. give people an opportunity to fail – change the finish line/assessment_
_Diversity of goals should be accepted fact, and supported by everyone
Lots of people working on many projects, some people working on just one projects, lots of dynamic teams _
_Common goal that everyone is working towards _
Someone applying to SWC can work on whatever they want: Here’s what everyone is doing, so pick someone, find the person, join your efforts
All projects are on some black board, and whenever a new scientist (irresp. of career stage) joins they can join any project, and switch between projects BUT with accountability (deliverables at the end of working on a project: protocol, report, preliminary data), flexible possibility to move on to either PI stage or staff scientist stage
Deliverables simultaneously provide incentive for open science in the building, since the continuity of project-based work relies on good reporting
_How to balance flexibility and commitment: more bureaucracy & structure; you need the meetings, but they should be concise _
_How to have flexible _
Multiple projects and one big question are not mutually exclusive
Orga for next meeting: (5mins)
Think about concrete proposals
Next time: DeepMind (Fred/Spencer) & IBL (Tom/Hernando) summaries
Meeting to be every other week on Tuesday, time tbc by slack poll
Spencer and Svenja to do facilitation/notetaking respectively next time
Ideas
**Todo (until
»»> gd2md-html alert: undefined internal link (link text: “next meeting”). Did you generate a TOC?
(Back to top)(Next alert)
»»>
next meeting)**
**Other places to interview / factfind **
Random Thoughts
#
Purpose of group
Brainstorming phase
Specific Ideas
Notes from Main Group meeting
##
Additional Themes
SWC-SWC + SWC-GCNU collaborations
Tech transfer within the building
Bring NRF to full potential
Why does the group exist?
Bias the SWC away from 1 paper per person (“The career success algorithm”), changing the scientific cost function
Fulfill the stated mission of team science and GCNU collaboration (easier said than done)
Build collaboration between groups within the SWC
Build collaboration between SWC and wider UCL/London research community
Build structures to facilitate collaboration on a multi-person level
Use SWC’s resource to model how we think academia should change
Sharing of ideas, concepts, techniques. Eventually broadcast that outside.
Devise strategies and mechanisms to get people talking, to increase research communications and ideas? (within SWC and across SWC-GCNU)
Share ideas publicly with wider research community
To encourage communication within the building? E.g. with fablabs from the beginning of the project.
Control animals in a more streamlined way, maybe doing more basic work that helps transversely across groups in the SWC. Basic research of animals’ wellbeing.
Highlight what work is going on in the building, encourage project-based work
Identify problems of motivation (e.g. among PhDs and postdocs)
Identify the ideal situation, then reverse-engineer what we can do at SWC to achieve that ideal
Research culture globally is defined by selfishness and competition
Work to make SWC a beacon for wider scientific community
Notes
Existing academic structure: 1 paper per person, limits what we can achieve
Research is career rather than science-question oriented
Get inspired from other institutes: Allen, Crick, DeepMind / Google, …
What would have more impact, being a scientist or changing research culture?
We have a lot of overlap with the culture working group– need to define clearly what our goals are about research specifically.
‘If we were not part of the rest of the world, what would be the ideal situation we would like to have for doing science?’
We need to define our Utopia.
Problems Identified
Grants – funding and constraints
What PIs do, how they use their time
Diversity of career paths in science
Management within groups, etc.
The challenge is translating a local model (at SWC) to the wider world. Or to use team science as a valid ‘currency’ to get a job elsewhere.
“Most of the negativity, if not all” stems from the first author academic structure
PIs suffer from the same career-driven issues as students
Ana Collated 29/10/20
Joaquin 29/10/20
10 Leadership Principles'' and they argue that they are essential for their culture. Shall we define SWC/GCNUxx Scientist Principles’’ that may help us define us as SWCers.Spencer (29/10/20)
Joaquin
1. The big picture
Team science and open science is the future for neuroscience research. It would be great to deliver publications presenting results that are unachievable in the traditional single-author paper framework.
This paper by Christopf Koch and Allan Jones from the Allen Institute nicely demonstrates the potential of this type of science, as well as the sociological experiment in building the type of science.
2. Proposals to enhance our (collaborative) research culture
2.1 Invite scientists that have successful experiences creating team science in experimental/computational neuroscience research environments to talk about their experience (e.g., things that worked and did not worked building their research environments) For example, we could invite Prof. Christof Koch to give a talk.
2.2 Create research-oriented working groups (Tom Mrsic-Flogel’s idea)
These working groups would be focused on a larger research problem. They will be composed by experimental and computational neuroscientists. These scientists will collaborate to solve the larger research problem. An example research problem is better understanding computations performed by neural populations. Experimental members of the working group could bring their population recordings and questions they would like to address about these recordings. Computational members would then propose methods that could best fit these datasets and research questions. Experimental and computational members would then work together using the proposed methods to characterize the population recordings, leading to joint publications. Another example research working group could center on the quantitative characterization of animal behavior (with or without simultaneous neural recordings).
2.3 Collaborative data talks
These data talks will present research among experimental and computational neuroscientists or collaborative research from neuroscientists from different areas. They will illustrate the power of collaborative research.
2.4 Reproducible science training
‘‘It has been claimed and demonstrated that many (and possibly most) of the conclusions drawn from biomedical research are probably false.’’ (Button et al., 2013). It would be great if at the GCNU and SWC we produced scientific results that are reproducible. To facilitate this, we want to train our staff on software tools that facilitate this reproducibility, like Python, Git and Anaconda.
SWC members have created the Python club (a journal club style meeting discussing topics related to Python programming) and PyStarters (a short introductory course on Git, Anaconda and Python programming). These resources are well attended by both SWC and Gatsby members. Besides their technical contributions, they help SWC and GCNU scientists interact with each other.
3. Proposals to enhance SWC-GCNU collaborations
3.1 Create a “data clinic” for SWC to link with GCNU
These data clinics could allow anyone from the SWC with a dataset link with someone from GCNU to discuss how to best model the data. These would be initially informal “starting points” but could develop into something more concrete. The frequency of these clinics could be monthly or termly - depends on how many people would be interested in this. Making it a regular event would make people think about getting advice - I (Ana) would be against an ad-hoc model.
3.2 GCNU-SWC PhD rotations
Encouraging GCNU to do rotations in SWC and vice versa is a good idea but perhaps it should be “encouraged” and not forced.
For example, in their first year inference course, GCNU PhD students learn about advanced inference methods. After this course they could spend a few months in the laboratory of (for example) John O’Keefe learning about social behavior in mice and applying switching-state Hidden Markov Models to quantitatively monitor the behavior of freely moving socially interacting mice.
Also, in their first year SWC students take the Theoretical and Systems Neuroscience course, where they learn about regression and dynamical systems methods. After this course they could spend a few months working with members of the group of Peter Latham applying what they have learned in this course and developing new methods to understand their neuroscience data.
3.2 GNCU-SWC joint journal clubs
Encourage SWC scientists to attend the GCNU Theoretical Neuroscience journal club. Stimulate journal clubs led jointly by GCNU and SWC scientists. It should be a great experience for both GCNU and SWC scientists to jointly prepare and discuss papers of mutual interest.
Does the SWC have center-wide journal clubs? If so, we could encourage GCNU to attend them.
First yr PhDs already have joint JCs during the first term - this doesn’t continue beyond term 1 though - we could suggest extending it?
Philip 03/11
Issue: Due to the grant system, even SWC PIs 1) do not necessarily pursue the science that they believe is most useful to the field in the long term; and 2) spend a lot of time pursuing grants rather than doing science/scholarship/teaching.
Potential solution: Negotiate (more) deals for institute-wide funding for research purposes. If SWC PIs are getting $X on avg from Funder Y each year anyway, this makes more sense than going through the grant system. Because this facilitates good science, Funder Y would benefit too. I imagine this is mostly a question of pursuing difficult/uphill/long-term negotiations that I know nothing about.
Issue: Topic+method-based (experimental) neuroscience is a norm and is taught to trainees by default. A common formula goes:
1) Pick a general topic, e.g. escape, reinforcement learning. Pick a task and an array of modern techniques (e.g. large-scale recording). Apply these to a relevant brain region(s).
2) Take the positive results, and piece together a broad, intuitively satisfying paper.
3) Once the trainee leaves, do not re-test in other contexts.
An alternative is science that is driven by a question/hypothesis/theory. For example:
1) Come up with a specific hypothesis you think is important. Set up/use an experimental paradigm that can address it. Adjust the hypothesis based on pilot data.
2) Test the modified hypothesis in a new set of animals/cells. Publish results, explaining the hypothesis, result, and assumptions.
3) Once the trainee leaves, falsify, generalize, or deepen these findings in a new project.
Potential solution I: Open discussion. Do people agree with that topic+method-based & individual experimental neuroscience is the default approach (not including methods/theory)? If so, would we like to shift toward question-based & team-based science?
Potential solution II: Instil the question-driven approach by requiring/training researchers to go through it. For example, ensure that project proposals/data clubs include specific, tractable hypotheses; justification that the experiments are the best way to address them; the framework for interpreting results; and all relevant assumptions.
Potential solution III: Enforced Gatsby-SWC collaboration. Recruit trainees to be coequal-author collaborations between a theorist and an experimentalist, starting from the planning stage of the project. Recruit a theoretical-minded PI to SWC and experiment-minded PI to Gatsby?
Potential solution IV: Get rid of the relationship between project execution and a need to publish a first-author paper in a top-tier journal. In my view, this pushes the ‘researcher cost function’ away from thoughtful, cumulative science.
- **IVa**: Team science / working groups (see other people’s proposals)
- **IVb**: Somehow implement a cultural shift such that which journal we publish in becomes an afterthought, and the question comes first.
- **IVc**: Design, fund, and/or point out an existing system for digesting published science that goes beyond subscribing to top-tier journals.
#
1 efforts:
- 6 monthly cycles
- hierarchy of efforts
- everything broken down on milestones
- big meeting with the whole company -- OKR day every 6 months
2 teams:
neuro, RL, (now theres a multi-agent team)
efforts draw from teams
manager will have expertise in your area
teams might have one person from an area
each person might be on multiple teams
leaders, contributors, advisor – in order of time commitment
how are efforts, etc determined/decided?
merge top-down and bottom-up deep learning
using games, deepRL, working in simulation
new memory structures will unlock a lot of planning
these big assumptions are coming from senior leadership
beginning in demis’ head, expand to senior leadership, etc
3 information flow:
company wide meeting every wednesday (!!) - 1hr
nominally not just research, but majority research
present recent results for 7-8mins
internal project management tools
cross-pollination, lateral connections
is success because of structure or culture?
these are connected, so they feed off each other
pumping positivity in is crucial to make sure things don’t slide
collective sense of growth, being well-resourced helps a lot
instances of people being competitive for intellectual disputes in good natured way
values
kindness, collaboration, mission-driven
vagueness means no real behavior change
might be more useful to prescribe behaviors and best-practices
maybe values first, then make them concrete
technical lead and program manager on a team
program managers support research teams
- communication
- interpersonal issues defusing
- mixed feelings -- two-class system emerges
Frequent meetings as a tool to create collaborations:
Evaluation of work / progress
Milestones
Meeting culture