2018 Symposium on Artificial Intelligence in Education

Multimedia Whitepaper

Matt Zucca, Tylor Burrows, Nicolas Rutherford, Radamis Zaky
JULY, 2019

Executive Summary

The E21 Consortium’s first annual symposium was held at the University of Ottawa on October 21-22, 2018. The topic of artificial intelligence and education was chosen, as AI technologies are set to revolutionize both educational contexts and the broader societies in which they are situated.

This report briefly describes the event, then presents an analysis of artefacts documenting the symposium in the form of videos, texts, and posters. Key themes explored in the data were:

The report’s conclusions directly address the four questions about artificial intelligence in education that were posed at the outset of the symposium:

2018 Symposium on Artificial Intelligence in Education Multimedia Whitepaper

In furtherance of the E21 Consortium goal of encouraging disruptive dialogue within education, on October 21-22, 2018, at the University of Ottawa, the first E21 symposium was held on Artificial Intelligence in 21stCentury Education.

Advances in computer technology such as raw processing power and the neural networks that drive machine learning, combined with newly available large data sets, have led to an explosion in ‘artificial intelligence’ (AI) applications at present and over the horizon. Medical image processing, automated personal assistants, and financial investment algorithms are already in use, while self driving cars and highly customizable robotics seem just around the corner. These advances will continue to be adopted across services, industries, and economies. For educators this poses questions about how to respond. How will artificial intelligence influence what is taught to students? How will artificial intelligence influence how students are taught?

Hoping to continue building a network around imagining education in the 21st century, the E21 Consortium invited a host of panelists, sponsors, students, experts, and other attendees to participate in their vision.

To guide the planning and give a direction for the symposium, such as panelist selection and symposium activities, the following initial questions were posed:

1. What configurations of power, economics and industry drive the design, development and deployment of AI? To what extent can institutions of higher education play a role in the harnessing of AI?
2. To what extent do the dictates and demands of the push towards ever-increasing development of AI – as played out in research and private sector interests – displace the traditional skills, knowledge and expertise that our institutions of higher learning continue to train and educate its students for? What societal safeguards might we need to create and implement in light of such a potential radical disruption?
3. What are some of the current benefits AI has provided to institutions of higher education in administrative and disciplinary applications?
4. How might we conceive of a more equitably-designed, developed and deployed AI in light of the rate and range of disruption some have forecast as inevitable?

PARTICIPANTS

The participants registered for the event included students from high school to the doctoral level, as well as educators, researchers, and other interested parties from the public and private sectors. Having input from a range of stakeholders and perspectives was considered crucial to the success of the symposium. Figure 1 provides a breakdown of participant numbers.

To address the multidisciplinary and complex questions posed at the planning stage, the panelists were selected from various disciplinary backgrounds including ethics, law, art, film, computer science, and management. Figure 2 displays the invited panelists, including their affiliation.

Rediet Abebe

Sofian Audry

James Barrat

Chris Dede

Stephen Downes

Nevena Francetic

Timnit Gebru

Abhishek Gupta

Ian Kerr

Matthew Mckean

Alastair Summerlee

Figure 2: Invited Panelists

This year’s event was organized and sponsored by several technology companies and educational institutions, as well as the City of Ottawa. The lead organizer was the University of Ottawa. Figure 3 shows the logos of E21 Consortium members, as well as sponsoring organizations.

Thank You To Our Consortium Members:

Our sponsors who made this event possible

Figure 3: Consortium members, and symposium supporting organisations

On the evening of October 21st a reception hosted by Shopify welcomed participants and panelists. In addition to providing a networking opportunity for participants, panelists in attendance were invited to give a brief introduction to themselves and to pose a question for participants to consider in advance of the next day’s activities (see Figure 4).

Figure 4: Questions posed by panelists as part of their remarks at the Shopify reception

The symposium’s main event on the 22nd began with the president of the University of Ottawa, Jacques Frémont, welcoming the panelists and participants, followed by an introduction by the University of Ottawa’s Vice Provost for Academic Affairs, Dr. Aline Germain-Rutherford.

The symposium activities began with three interactive panel discussions, each comprised of three panelists and a facilitator. These panels, which each lasted 75 minutes, were supplemented with pre-recorded video clips from remote panelists, and began with the panelists speaking to a specific topic with audience questions and comments woven in by the facilitator. Audience participation/engagement was further promoted through a Twitter hashtag (#E21sym) and the collaborative online tool Nureva Wall (see Figures 5 and 6), with both displayed in the symposium venue (see Figure 6). The panels were live streamed and recorded.

The three guiding probes were as follows, although the facilitators each chose to ‘disrupt’ the question with minor changes:

Following the interactive panel discussions, two ‘disruptive dialogue’ sessions were held. These sessions provided the participants with an opportunity to imagine the role of AI in education by using flip-chart paper to design an educational robot on paper.

Participants were first asked to identify an education problem and create an AI robot to solve it. They were to: Name it, Draw it, Label key features, and Explain how it fixes the problem. After this, they: Shared it with their team, Used stickers to highlight great features on any robot, and Took a picture to post on Nureva.

After the individual poster session, a team robot was created. This time, participants were asked to: Identify a wicked education problem, Co-create a Team AI Education Robot, Name and draw it, Describe its features, Pretend it was human (what would it feel, see, hear, sense?), Use stickers to identify favorite features, and Pick a team representative to present at a plenary at the end of the second disruptive dialogue session.

The final session of the day was a statement from panelist Stephen Downes which opened up, extended, and clarified some of the key ideas that emerged during the symposium.

Between each of these activities was a short break providing participants the opportunity to network as well as appear in video-interviews regarding the symposium, using the ‘vox-pop’ format.

The symposium activities yielded a varied and generous data set including:

• Videos of the interactive panel discussions
• Videos of interviews with remote panelists
• Vox pop videos of participants
• #E21sym tweets by participants
• Nureva Wall posts by participants
• Posters created by participants

We collected these together and, after an initial scanning of what had come out of the symposium, established a thematic frame which could incorporate the breadth of materials collected.

Yet there were conflicting views surfacing online. Throughout the presentation participants were doubtful of the usefulness of the comparison made between human beings and machines. Some Nureva users wondered “AI as a metaphor for everything that is not human… how useful is that?” Meanwhile, a Twitter user suggested:

Furthermore, participants were sometimes divided on what it means to learn, to know, to teach, and to educate. But most agreed that critical thinking was extremely important for today’s and tomorrow’s students. The “importance of teaching children critical thinking and theory of knowledge,” as well as the belief that “AI cannot replace education in critical thinking” were widely affirmed and repeated. At the same time, some participants questioned how easily critical thinking skills can be efficiently taught. Facilitator and panelist Alastair Summerlee was quoted on Nureva saying “don’t prepare us for jobs, train us to think and feel!” Others who were critical tried to pin down exactly what a creative skill was and how it relates to AI. One idea to incorporate AI in education was to “Give kids problems so they realize why, and when, they need to use AI #E21SYM.” During the first panel Summerlee confessed his long-time ambition of destroying undergrad studies. Afterwards, participants expressed dissatisfaction and concern about today’s educational system.

There was a small but repetitive current of posters to Nureva who were unsure of the effects of this new technology on humans. Some argued that “technological dependence leads to decreased creativity,” while others were more focused on its effect on children. They doubted that AI could support brain development and asserted that “there is plenty of research on the impact of technology on the brains of children growing up. It can affect their brains. Tech is good but [for] children?” Others were sure that they “don’t want AI to ‘decide’ what I learn.”

Regarding teaching, some participants believed that “teaching must lead to inspiration” while others were uncertain whether teachers’ presence was needed since face-to-face contact with teachers is possible online. Yet, the low completion rate of massively open online courses was cited as an example of the failure of online education initiatives. Someone went as far as saying that “We are not training children. We are training tomorrow’s adult cyborgs.” There was a desire for “AI [to] be used to broaden schooling for those that struggle with standardized testing.” Learning styles and methods of teaching were brought up in response to the supposed failure of standardized testing to properly measure aptitude. In some cases AI was seen as the solution to aptitude testing, while in others it might only exacerbate its flaws, arguing that “AI grading might accept ‘correct’ but not diverse answers.”

Ultimately, the ambiguity of the term artificial intelligence and what it is capable ofled to many conflicting questions and concerns, several of which were not addressed.

The development and deployment of AI into many sectors and fields today is fueling some concerns, notably about its effects on employment and inequality. It also invites general questions about ethics and the risk of harmful biases being programmed into AI machines. AI systems that are underpinned by deep learning algorithms depend on data being fed into them. When this data is being selected by humans, the potential for biases being passed on is inherent and should be monitored closely.

Symposium participants were adamant that having a diverse team of people creating artificial intelligence is superior to any other arrangement. It was argued that “Questions, solutions, and impact will depend on the background and experience of the scientist(s) doing the work, so diversity is important.”

Several times during the discussion of this topic participants, both online and off, criticized the symposium as well as the development of artificial intelligence as being biased in favor of ‘Western values.’ They suggested things like “AI was invented in North America, ergo Western perspective of the tech,” and became skeptical of the ethics that would be built into it asking “Whose ethics would the AI be based on if people’s views on ethics differ?” and “Are we going to be able to create an AI that is unbiased?”. Panelist Stephen Downes suggested that the problem of ethics in AI might be doomed from the start by saying “Our big problem with AI and ethics is the fear, a genuine fear, that an AI might act like we do when it comes to ethics and it might make the kinds of ethical decisions that we already make. So, I think we need to think of ethics and how we are doing that as a species before we start thinking about ethics in AI.”

To mitigate the problem of bias in AI some posters and speakers suggested that a diverse background of developers in AI will be able to represent different interests, as well as to see things that are normally unseen. They were anxious about the representation of the marginalized, who, it was widely felt, AI could leave behind.

For example, some worried “Where is the dialogue referencing those with no www access?”. On the other hand, some users thought that those posters were confusing access to the Internet with the problems of artificial intelligence. Yet, posters seemed to support the idea that “The Internet is the gateway to the modern economy, education, information.” Meanwhile, others thought that “If the data is available to everyone” there will be nothing to worry about. Many wondered if AI will “erase the ever increasing chasm between the have and have-nots?” Panelist Chris Dede gave an example of how this might happen, and how educators can try to stop it:

Dede thought of advanced technology like a ‘sorcerer’s apprentice’ which people should learn to use, rather than to compete with. He thought that by teaching people to use this sorcerer’s apprentice we could avoid widening the gap between the haves and the have-nots. In the case of the super GPS, it is only a stand-in technology that provides a hypothetical case where someone with a better understanding and easier access to an advanced technology could make better choices for themselves.

Others anticipated unpredictable change and possible benefits as well as drawbacks. They implied that jobs may not ‘disappear’ but change, and the idea was often repeated that AI could “create new jobs that we cannot fathom: think of web devs pre-web era.”

One of the largest challenges that participants believed we would face was an AI-created apocalypse scenario. Though panelist Ian Kerr was ‘cautiously apocalyptic’ he and many others were focused on the positive potentials and more conservative drawbacks of artificial intelligence. However, despite these assurances, people continued to bring up their fears. They argued that ‘AI will develop because of Defense,” and that “bad actors are presently working on weaponizing AI”. Geopolitically, the “western countries are against the eastern ones in developing AI. Cold war” and “AI could be a winner-take-all race”. Many people were concerned about the international consequences of AI development for military use and asked “If no one is attacking you why increase your defense budget?”.

To tackle current and upcoming challenges in AI posters felt that “AI is not the same as other domains and needs its own solutions,” yet it is “being foisted on society with no democratic discussion.” They wondered if “there [were] political parties tackling AI realistically?” and proposed that “a diverse team of ppl can make decisions. It’s just very expensive vs AI.” On Twitter people asked the panelists directly about the circumstances of machine-guided decision making, as well as the lack of oversight in existing deployments of AI in the private sector:

Noticing that AI was developing so fast that it could outpace democratic discussion one participant explained:

Panelist James Barrat drew attention to the challenges facing the connection between employment and education. He put forward the idea that jobs would be lost or changed, and suggested avenues for students in light of those expectations:

Others were more uncertain, “the point is whether AI is going to remove teachers and even learners or?” Some, assuming educational institutions will continue to exist, focused in on computer science. They wanted to know if ethics classes were mandatory for computer science programs, and someone suggested that “training on ethics and society lags behind technical training.” The consensus was that the education system is outdated, and many people echoed the statement that “Most teachers that have been teaching the same curriculum for a long time — are using the same things ie. tests and it should be more diverse.” Exacerbating the problem, however, was the recurring idea about what counts as knowledge, as well as how to identify something that works. As Stephen Downes pointed out at the end of the symposium:

How can AI help students think outside of the box? Overall, people felt that education itself should be more personalized, it should take into account student’s emotions, and it should help students think outside of the box. It should be affordable too, else many schools will not have access.

Ian Kerr urged educators to come up with a way to decide when, where, and why to use technologies in classrooms, lest they run into the problem of PowerPoints:

Finally, there were a few statements and posts that didn’t fit easily into one category, or spanned so many that they deserve an honorable mention. For instance, Chris Dede was especially critical of what he saw as a failure of institutions, especially education, to use the research that was already out there:

Others brought up long-standing and critical philosophical problems such as: the difference between right and wrong, who defines what is right and wrong, sense and nonsense; the idea that “religious and secular ethics are products of human nature so the difference ultimately does not change much”; and the considerations of a “Consequentialist versus Deontological point of view.” These kinds of posts appeared every once in a while and went unresolved. Others discussed whether it was possible to accurately predict the future, even with the help of artificial intelligence, whether code had ethics that were identifiable before its deployment, or the identification and consequences of ‘errors’ made by an AI.

The benefits of AI in education were not highlighted by panelists and participants as much as concerns and challenges were. However a common vision of the (potential) benefits of AI did emerge between the lines.

Many participants believed that the emergence of AI in society not only meant that new skills would have to be taught in schools so as to best prepare students for the future, but that bringing AI inside of the classroom would also foster new and better ways of teaching and learning. Several participants expressed a tempered hope that with AI tools, we might achieve better teaching and learning outcomes, in part because we’d move away from standardized testing models – “AI should be used to broaden schooling for those that struggle with standardized testing” – and allow for more effective personalized learning, for example. This would in turn better prepare students for the job markets of tomorrow, or empower them to invent new occupations altogether. For participants, though, there is still much that will need to be figured out: “Teaching must lead to inspiration. How can AI contribute?”, and “How can AI help us change traditional classroom settings?”.

There seemed to be a general feeling among participants, especially the students, that AI could and should be beneficial for society: it could “solve social problems that still exist, unaddressed” and “[…] AI needs to help solve collective goods problems”. On a macro scale, there was even hope that it would address some of the inequalities that persist between the global North and South, where access to the internet and its associated learning potential, is still limited. One nureva user proposed, in terms of learning, that “AI can be used to create toys that teach children in developing countries”. But several users pointed out that many of the issues that poor and marginalized people face abroad, and in Canada, could be addressed right now, without AI tools: participants and panelists were clear in stating that AI is not a panacea, despite the great potential it carries. It cannot, and will not, solve every problem that people or communities face.

Remote panelist Abebe pointed out that AI techniques could help to reduce inequality, while at the same time remote panelist Gebru added that AI can exacerbate inequality or provide opportunities, depending on how it is implemented. In order for everyone to benefit from AI in education, being mindful of inclusivity and representation in the development and deployment of AI tools should be at the center of any action.

During the “disruptive dialogue” activity, participants were asked to draw their ideal AI robot in education and label its key features and special abilities.

In total there were 134 references to AI applications included on the posters. The three main applications were personalized learning (49), emotional support (14), and logistic support (10).

Participants showed considerable interest in the type of AI robot that “teaches every student in a unique way, tries to understand every student’s point of view”. Examples of this were varied, and were succinctly if inadvertently summarised by one participant who envisioned a system that:

One example of personalised learning supported by an AI robot also fosters inclusionary education, specifically that it “Provides learning assistance specific to disability”. Personalised learning extends to motivation, with participants interested in AI robots that could “identify when students are bored by reading [students’] body language”, or relates what students need to learn to their actual interests and real life situations.

Personalised learning also applies to assessment, which is an area of education already  achieved with computers. Simple technology such as multiple choice are now accessible to anyone with access to an Internet computer, and AI-powered language processing is being tested on student writing.  The ability to “customize assessment to the learner” is both related to personalized learning and to automated assessment generally which became a sub theme also.

The second most prevalent type of application was one which could provide emotional support. One of the posters to Nureva pointed out that, in the future, “AI can/will be capable of so-called human traits like creativity, empathy”. This is likely because, for example, facial recognition software that gauges mood is already under development. One of the participants’ posters incorporated this and other forms of emotional support with a robot that:

Other descriptions of emotional support-enabled AI educational robots included that they would be “Friendly, safe” and “100% confidential, will talk to you about any issues you have in life. Will always be there for you”.

Finally, logistic support, while not directly related to pedagogy, is a crucial aspect of education that the participants felt AI robots could support. From using “advanced algorithms to create unique solutions for transportation costs” to providing essentials like food and pencils, to help finding income or “services (e.g. medical, therapy)”. An AI robot which “deals with bureaucracy” is a suggestion which highlights a burden that has counterintuitively gotten heavier for both students and educators in conjunction with the rise of supporting technologies.

Other specific applications of AI in education encapsulated in the posters included automated assessment, teacher assistant, and experiential learning.

Solutions which participants brought up were primarily related to AI development which is inclusive of diversity. In terms of existing solutions, remote panelists Abebe and Gebru both belong to and spoke of special interest groups which are focused around minorities and aim to promote minority inclusion in AI development. These groups may be interdisciplinary and data-driven, and can augment existing AI-oriented research labs and masters programs by and for the developing world. Examples of activities conducted by these groups in order to bring together and support diverse and geographically disconnected community members include a Facebook forum for support and communication, conference activities such as workshops designed specifically for minority groups to showcase, share, and network.

Another existing solution for reducing bias in the development of AI systems was shared on the Nureva Wall. This was a description and link to a machine learning crash course module offered by Google and “which looks at different types of human biases that can manifest in training data, and provides strategies to identify them and evaluate their effects.” A link was provided for interested parties to learn more. The availability of this module is a helpful step towards helping engineers to consider ethical issues more deliberately, a proposal put forth by remote panelist Gebru. Remote panelist Abebe also recommended that developers keep ethics in mind, as well as engaging with other disciplines and being aware of the diversity existing in their local spaces.

Gebru expressed the need for more diversity in the data sets used to train AI, an issue which was highlighted by the October, 2018, disclosure that Amazon decided not to move forward with developing an AI recruiting system which was found to be biased in favor of men. In order to evaluate datasets, Gebru and colleagues have written a whitepaper which aims to encourage those who are releasing data sets to write documentation about those datasets (e.g. distribution, intended uses) to help users determine if a data set should be used for a situation. The whitepaper is available here. As mentioned in the ‘Challenges’ section above, one user opined that “AI is not the same as other domains and needs its own solutions,” however, Gebru proposes that AI developers can learn from other industries about how to address problems and put in safeguards before AI technology is too ubiquitous.

The E21 symposium on education and artificial intelligence provided participants with opportunities to contribute to the discussion through online posts, panel discussions, and poster creation. Whereas the thematic findings, above, cover the breadth of the symposium, by considering underlying contradictions in the themes a conceptual map of deeper meaning emerges.

What did participants try to communicate about artificial intelligence in education? Which questions, ideas, and desires were participants trying to express? There are no easy answers to this question, so we did several concept maps and contradiction maps until we arrived at something that seemed to make sense and change how we thought about the data. We mapped the concepts that we thought people were talking about as well as our own concepts as we worked through the data. We also broke down every unit of data into ‘contradictions’ (e.g. Access/No Access, Employment/Education, Custom/Standardized) until we arrived at unresolvable contradictions in the data. Doing this created a lot of questions and new ways of thinking about what people were communicating about artificial intelligence in education.

On the surface, artificial intelligence in general was central to participants’ communications, rather than a consistent and specific focus on AI in education. Although many concerns did not seem directly related to the issue of AI in education, after reconsidering the data, it becomes clearer that participants were trying to situate artificial intelligence in education. Taken together, we can think of the collection of our data across media as a complex, multi-faceted discussion about artificial intelligence in education. What follows is a discussion and synthesis of the recurring, potent, or provocative imaginings from the E21 Symposium. Figure 9 illustrates the concept map which emerged from breaking apart and re-assembling the data.

Our analysis identifies three key figures responsible for AI in education: the stakeholders, the classes, and the interface (depicted in yellow on Figure 9). The stakeholders are the people responsible, in varying degrees, for how AI is integrated into education. The classes are the key educational tool for stakeholders, one of the central locations of the interface, and a key concern with the introduction of AI in education as it relates to the reconfiguration of social classes. The interface represents the way the technology of artificial intelligence is built, accessed, and distributed, and is a site of speculation and concern. Each of these key figures raises a corresponding question, which reflects participants’ differing perspectives and understandings regarding AI in education: What kind of stakeholders? What kind of classes? What kind of technology? The questions are depicted in purple on Figure 9.

We also explored the way relationships between stakeholders, classes, and interface were imagined (in orange on Figure 9), including questions and demands that participants raised. A feedback loop between the stakeholders, the classes, and the interface seems to be the main mechanism of AI’s participation in education.

Stakeholders include, but are not limited to: students; educational administrators; researchers in fields such as education, technology, and the social sciences; instructors; developers of AI technologies; government and policy makers; non-governmental organizations, and; other interested parties. It is difficult to distinctly identify and label stakeholders, because both education and technology affect, and are affected by, multiple interacting groups and individuals among and between societies and across the globe.

What kind of stakeholder can we envision for AI in education in the 21st century? Our analysis suggests that stakeholders be activists, since participants were fearful of negative outcomes of AI in general, and AI in education in particular, and they wanted those involved to be working for the interests of the students and society at large. Stakeholders must also be adaptable, due to the pace of change inherent in the world of technology and education; stakeholders and their organizations have to adapt to different sociocultural circumstances and contexts. As adaptable activists, stakeholders are ready to disrupt the status quo; they have to be willing to let go of traditional systems of education. Stakeholders need to continually expand and reach of their organizations so they do not leave people behind. The first and most obvious way that participants wanted stakeholders to expand was by improving access to educational services and material. One of the suggested ways of doing this was to improve access to the Internet. One of the stakeholders’ goals appears to be to address gaps in students’ ability to use the interface. When doing the contradiction mapping, we discovered that instead of the distinction between the haves and the have-nots, the distinction between those who are able and those who are not able better represented the concerns among participants.

Classes and the structure of classes emerged as a key figure of AI in education. This incorporates classes as both an educational tool and classes as a social object. As an educational tool, classes include not only traditional classroom teaching and learning contexts, but also online learning, home schooling, and other non-traditional formats. Socio-economic class is inextricably linked to learning contexts because inequality was seen as built into educational opportunities, especially where access to education propagates or exacerbates social inequality.

The kind of AI-supported educational contexts our analysis of the symposium suggests for the future is one that fosters socio-economic equity. Whatever form AI takes, and however it is implemented for educational purposes, participants wanted classes to be growth oriented and inclusive of diversity. This means that students will be empowered to realize their personal potential and that they will be shown empathy all while fostering human connections. The classes are focused on cherished values, though other than diversity and inclusion, no specific values were discussed. We are not capable of a meaningful assessment of implied values using the data and knowledge at hand, and as discussed earlier there are concerns about the dominance of traditionally ‘Western’ values, as well as questions about which ethical standpoints among many to guide policy making and technological development.

If classes are the main tool for education, then they should be a high priority for stakeholders. For example, teachers were often seen as the center of the classroom and they were the supposed executors of new educational techniques. Participants wanted stakeholders, especially through the administration of classes, to act to reduce inequality.

Diversity is one of the most sought after outcomes of AI in education according to participants. Across the board, participants wanted to see diverse measures, backgrounds, and relationships, aiming to reduce inequity.

Stakeholders need to make several important decisions regarding how different technologies will be used, and some of those technologies will be the main way students engage with their education. We call these technologies an ‘interface’ between the students and their education. The manner of interface may be a personal device, a cloud service, accessed centrally by multiple individuals, or something unforeseen.

Rather than interest in the manner of interface with AI, our concern here is with the kind of technology we should use. Participants demanded that the minimum requirement for technology to be used in an educational context should be that it is built and implemented by people with diverse backgrounds in terms of education and life experience. Technology should thus empower the students and the stakeholders. One of the ways technology could empower these groups is by reducing bureaucracy. However, what remains unknown is how to determine a way of assessing technology beyond how it is made and implemented. Still, the participants expressed that it is an important task to look at technology from many angles.

These technologies are seen as so important that their effects on the students and the stakeholders and the quality of education has to be vetted from every angle before being incorporated. One of the ways stakeholders can vet technology is by looking for indicators that this technology acknowledges diversity in how it was developed and how it can be applied.

Class interacts with AI through a technological interface. The interface should be built with constructive and collaborative sharing in mind. This includes sharing the opportunities to profit from AI in education as well as sharing knowledge across backgrounds.

The feedback students receive should reflect their strengths and be able to see what students are doing well. This was in stark contrast to the perception of standardized tests and other traditional tools for assessment. Reversing the flow of feedback, data acquired by the stakeholders and the interface should be able to incorporate diverse measures of success and be respectful of privacy.

• The Stakeholders are Activists
• Technology must be Vetted
• Technology Originates from Diverse Backgrounds
• Feedback Incorporates Diverse Measures of Success
• Diversity is a Primary Concern
• This Activity Must Reduce Inequality
• Artificial Intelligence and Data will be Used

The seven expectations, depicted in red on Figure 9, emerge throughout the data analysis and emerge as agreed upon outcomes for AI and education. These expectations, which cut across the following conclusions, are:

The authors of this report would like to thank the E21 Consortium members for their work as they improve education for the 21st century. We would like to thank the many sponsors who supported and continue to support the improvement of education with the E21 Consortium. Special thanks to the participants and panelists for sharing their knowledge and insights with us, may it continue to help us grow in the future. An extra special thanks to the organizers for their indispensable work, in no particular order: Marc Villeneuve, Gisèle Richard, Emilie Newman, Marc Bélanger, Manon Drouin, Chrystia Chudczak, Andrew Barrett and his team at Shopify, Carleton students and employees and Shaily Zolfaghari who volunteered at the reception. We also want to thank Patrick Labelle for his support in data analysis.

Last but not least we’d like to thank Aline Germain-Rutherford and Richard Pinet for their inspiring leadership.