Open curricula and assessment tools workshop in Kisumu
In August, Brendan and I attended the American Institute of Mathematics workshop “Open source mathematics curriculum and assessment tools”, hosted by Maseno University in Kisumu, Kenya. The workshop was a fantastic opportunity to meet new specialists in a variety of domains and hear about what they were working on. This post is a short summary on the workshop and what we learnt.
Last month, Brendan and I had a week together face to face. In and of itself, this would have been a lovely experience and a good chance to discuss all things Topos, but it was made even better by the fact that it happened during a fantastic workshop in Kisumu, Kenya, hosted by Maseno University, and organised by the American Institute of Mathematics (AIM). I’m going to write about our week and what we were doing there in particular, but you might also want to look at the official workshop announcement or the short summary by Maseno University.
Those familiar with AIM workshops might be surprised to hear that this didn’t take place at their usual venue at Caltech in California. In fact, this was the first ever AIM workshop to take place outside of the USA, and Maseno University were fantastic hosts. When planning the workshop (whose purpose I’ll get to shortly), the organisers pointed out that over half the participants were based in Africa, and had a pretty large overlap in terms of participants with the 2nd African STACK Conference for Undergraduate Mathematics which took place the following week in Nairobi, so it made logistical sense to host it in Kenya. But arguably more important than this, by hosting it in Kenya it suddenly became considerably more accessible for participants all across Eastern Africa, who represented institutes and communities substantially involved in the problems that the workshop aimed to discuss.
The format of AIM workshops is pretty fun for anybody (like myself) who has only really been to “traditional” maths conferences before. Instead of a day full of talks, there are only two talks each morning, and then the rest of the day consists of small to medium working groups. You can change which group you’re in for each session, or you can stay in the same one. This meant that there was a really fluid movement of discussion, with different groups intermingling, merging, dividing, and building upon one another throughout the week. Adding to this somewhat unpredictable structure was the fact that most of the speakers were only asked (or, really, politely told!) that they were giving a talk the day before they were scheduled to speak — an aspect that Brendan and I found out about on the Thursday. All in all, I didn’t really get much of a chance to sightsee because quite literally every waking moment was spent in discussion with other participants!
What was the purpose of the workshop?
In the past 25 years, Kenya has gone from having around 5 public universities to having 30. The reasons for this growth are many, and I am definitely not the right person to be giving a history of the political context of education in Kenya. But a key point to make here is that this growth has actually meant that class sizes have grown in universities there. For example, Maseno University has one of the strongest schools of Education and of Mathematics in the country, and many of the students of education have to take compulsory maths courses. This means that the classes are big. I spoke quite a lot to a lecturer who teaches five or so courses each term, with no TAs, and each class has between 500 and 1500 students. This number is really quite incomprehensible to many university lecturers in Europe and the USA. I remember teaching classes at Aix–Marseille that were considered “medium-sized” and had around 40 students, and if they had been much larger then I would have had support from a TA. Not only this, but the classrooms themselves do not make things easier: they are large rooms with lots of chairs, one small whiteboard, and no real internet access.
Aside from how hard this makes the practical aspect of actually teaching the classes, this means that one lecturer has to mark around 1000 final exams, for each course, by hand.
An “obvious” solution is to put all the final exams online and automate the marking. But how can this be implemented in a context where most students don’t have a laptop, and even those that do don’t have a stable internet connection? The most likely access to internet is through a mobile phone, but one that might be running a very old operating system, and shared within a single household. And then, even if you surmount these obstacles, how do you ensure that the exams that you’re setting are just as good as paper ones that you can set? If we want to automate marking, then it could just reduce us to mere multiple choice questions — how can we test for the ability to write proofs, or to draw graphs, or to do anything else that forms a part of the core skills that we should be developing as students of maths? The main focus of this workshop was dealing with the second set of problems, due to the relevant expertise of the participants; the first set of problems (those of technological infrastructure and access) is one that, I would argue, should be considered by a group led by different experts than maths educators and researchers in maths education.
One thing that we heard over and over during the workshop was the fact that these problems have led to a real opportunity to innovate, and that there are people working on good, ground-tested solutions, which are being led by the stakeholders themselves. This was reflected in the participants of the workshop, who fell into five groups:
- maths educators
- institutional policy makers
- researchers in mathematics education
- researchers in mathematics
- education-focused software developers.
When it comes to technological solutions, there are two key pieces of software that were represented at the workshop: STACK and WeBWorK. I’m not going to spend time going into these, because, again, I’m not a specialist in using online assessment tools for teaching. But the brief overview is that WeBWorK is a platform for lecturers to write questions and solutions and collate them into tests, and for students to take those tests in their web browser and receive feedback. STACK provides just the ability to write questions and answers, and is designed to plug into some other platform (such as Moodle) that handles all the exam collating and feedback stages. Each one has its own pros and cons: most notably, WeBWorK has a massive online bank of pre-written questions and answers, collected into courses; STACK has a full computer algebra system under the hood, allowing for the questions and answers to be really rather complex with randomly generated aspects and the ability to recognise mathematically equivalent but semantically different answers as equally correct. Both systems are used throughout the world, with STACK being used quite a lot in Kenya in particular. In coming to the workshop, we already had an idea of some of the problems that arise from the use of these tools though.
Technologically: how do we make shared resources available to institutes using different software? If one university adopts STACK and another adopts WeBWorK, then can we automate the conversion of educational materials between the two platforms so that institutes can collaborate?
Culturally: how do we best localise any materials to fit different curricula? Maybe more importantly, and something that many local participants were very keen to stress whenever I spoke with them, is the question of how to localise materials culturally, so that Kenyan students don’t feel like they are merely “consuming” American content?
Financially: how do we support the ongoing costs of free open-source software? Even if software itself is free, the infrastructure required to self-host is often non-trivial, costs money, and requires expertise to set up and maintain. Not only this, but effective use of any software requires time and training, neither of which are “free”.
Structurally: how can we help educators explain to their institutes how and why to implement these technologies? Part of this ties back into the financial question above, but it also needs an understanding of how individual universities decide on budgeting and training, and thus the context in which they make these decisions. In other words, it’s likely not enough to just make lots of great tech demos and user-success stories.
Educationally: how can we enable findings in maths education research to end up guiding the writing and development of educational resources? Some findings are incredibly context specific, and others much more general, so how do we reflect this when updating shared resources between different contexts?
In fact, around 50 discussion questions were raised on the first day of the workshop, and nearly all of them focussed on the non-technological aspects of technology use; not one of them was of the form “how can we use more technology to solve this problem”, and any that asked “how can we make a certain technology better” always placed it within a specific context, not suggesting that there exists some universal standard of “better” for any given technology, no matter how specific its intended use might be.
It is important to note that this workshop was a “working workshop”, i.e. not one where we expected to find answers, but one where we aimed to find paths towards answers, and to build collaborations to explore these over a longer timescale. Even so, there were some pretty incredible immediate achievements that came from this workshop, the design and promised implementation of an entirely new multidisciplinary masters course for a university in Kenya not least among them. But these are not my stories to tell; I hope to be able to point you to write-ups of these outcomes as time goes on and the groups that we formed continue to work together. For now, I’m going to finish by talking about why Topos was there and what we spoke about.
What were we doing there?
A big theme of the workshop was interoperability, allowing for sharing and collaboration across institutes using different technological solutions. One conversation that I repeatedly found myself in during the week was about how we can “translate” resources, e.g. from a STACK question to a WeBWorK one. Part of a solution to this might look like designing a custom domain-specific language that has the maximal shared expressivity of any two given formats, and this is something that we’re thinking about here at Topos. Something slightly more large scale, however, was an idea of multi-layered ownership and structured version control.
Brendan and I gave a talk with the snappy title:
Multi-layered Ownership of Structured Artefacts. A vision for international, contextualised cooperation on developing and evolving evidence-based mathematics curriculum and assessment materials.
This is a topic that we’ve been talking about, in collaboration with IDEMS International, for a little while now. The most fantastic opportunity presented by this workshop for me was getting the chance to present some ideas to this specific group of people and hear their feedback — was this something that they found exciting? That they felt could be community-led and locally maintained? More fundamentally, had we accurately understood what the actual problems that they wanted to solve were?
I’m not trying to write this post as a cliffhanger, but I don’t particularly want to go into the weeds of what our talk was about in this post, because I’d like to write something more detailed in the future, and I’d rather not explain something at all than explain it badly. If I had to summarise the talk in a glib motto, it would be:
Co-operation requires finding common ground, while respecting differences in context.
It is this second part of this sentence which proves to be, I argue, out of reach of current collaborative technologies. Brendan put it in rather more sophisticated language by saying that
Cooperation and mutual understanding is built by negotiating meaning together, and to assist with this, we build tools that help people:
- Capture epistemic frames and their relationships
- Own their understanding by expressing it within that epistemic frame
- Re-examine their roles and identities as collaborators in constructing an artefact.
Again, I’ll (one day) be writing more about what this means in the specific concrete example of shared educational resources, as well as in other settings, and I look forward to listening to what you all have to think about this.
