Datasets and Bias

Paradigms of AI usage in Higher Education: According to Ouyang and Jiao (2021) there are three paradigms of how AI is being utilised in education. AI-directed, where learner is considered as recipient (paradigm 1), AI-supported where learner is perceived as a collaborator (paradigm 2), and AI-empowered, where learner contributes as a leader (paradigm 3). Paradigms 1 and 2 have been the focus of AI education in the past two decades. However, there is a need for Paradigm 3, i.e., AI-empowered, Learner-as-Leader to be enacted in Higher Education that is centred upon promoting human intelligence and AI to be integrated, so that issues of biases in AI algorithms and datasets, lack of governance and how an AI system makes decisions and offers outputs for learning and teaching experiences can be more socially just and inclusive.

Types of Datasets: For learning and teaching experiences to be socially just and inclusive regarding the use of AI, it is important to understand how students use the AI platforms and what forms of information do they input or are provided with as an output. This involves use of datasets that are text, image, audio and/or code based. In BE education, students can engage with AI platforms using either of these types of datasets depending on the subject type and thematic topic/content under focus for their learning.

Following is a basic description of different types of datasets:

Image Datasets: Generative AI heavily relies on image datasets to train and generate new visual content. Images are typically classified, detected, and segmented using image datasets that consist of large collections of labelled or unlabelled images. Generative AI models are developed for image synthesis, style transfer, and other creative purposes with the help of these datasets.

Text Datasets: Generative AI text datasets are an essential component of natural language processing (NLP) models. These datasets are collections of text data that are used to train artificial intelligence models to generate coherent and meaningful language.

Audio Datasets: Audio datasets are used to train generative AI models. These datasets are used to generate audio content such as music and audio synthesis. Datasets include collections of audio recordings, including single sounds and full-length songs, which are used to train machine learning models to produce new, original audio.

Tabular Datasets: From financial analysis to predictive modelling, tabular datasets are frequently used to train generative models. In tabular data, data imputation is a common application of generative models.

Datasets can be grouped together and may lead to several outcomes depending on whether data in one form leads to an output on an AI platform in another form such as shown in the diagram below (this is a simplified version)

https://www.cogitotech.com/generative-ai/

Depending on the type of the AI platform such as ChatGPT, the text datasets may not be carefully curated or selected for the AI platform to produce outputs relevant to cultural, linguistic, ethnic, and historical backgrounds of the topic for learning or the socio-cultural background of the student.

Data Sets usage/role along the paradigms: Datasets in Gen AI can be used for textual outputs such as essays, reports, summaries, reflective narratives, thesis, rendered images, development of audio outputs and drafts. Gen AI can be used for data analysis including learning analytics and data from LMS.

Types of biases/exclusion points in Datasets: Algorithms in AI platforms and systems/Machine Learning systems that seek to do things better by increasing efficiencies can embed existing biases and replicate existing conditions of disparities. Large social datasets feed systemic bias into algorithms, and unchecked algorithms can result in systemic discrimination that favours certain individuals or groups over others. Machines rely on training through use of large amounts of datasets in different forms such as text, images or audios to recognise and produce outputs in response to prompts entered or the nature of task demanded. Most datasets are Western-centric because of the dominance of these forms of information that are readily available for the Gen AI platforms such as ChatGPT to utilise. This can hinder the achievement of social justice in classroom and decolonisation efforts in place by higher educational institutions.

"Although AI can assist in creating substantial and insightful content, it is not a simple task and demands proficiency in both technical skills and knowledge of ethical considerations. Students need to be made aware of, for instance, stereotypical ChatGPT answers. As ChatGPT is trained on a large dataset of text, it is very likely that the data used to train ChatGPT may contain societal biases (Dahmen et al., 2023). Consequently, the model will reflect these biases in the output data and, thus, reinforce existing societal issues and discrimination (Atlas, 2023). Therefore, when using ChatGPT in higher education, students should be aware of this potential bias and critically reflect on each statement created by ChatGPT (Gimpel et al, 2023). " 

Ferrara 2023,

When asked ChatGPT, ‘ChatGPT do you use datasets that project exclusion and promote bias?’, its response was as follows:

As an AI language model, I don't have direct access to information or knowledge beyond what I have been trained on. I have been trained on a mixture of licensed data, data created by human trainers, and publicly available data. My purpose is to assist and provide information to the best of my abilities, regardless of biases that may be present in the training data. However, it's important to note that I do not have control over the training process or the selection of the data I was trained on.

OpenAI, the organization behind ChatGPT, is committed to addressing biases and promoting fairness in AI systems. They have guidelines in place to prevent the use of datasets that explicitly promote bias or exclusion. They strive to provide a system that is useful and respectful to users from various backgrounds.

If you come across any biases or concerns in the responses I provide, please let me know, and I will do my best to address them.

Role of the ‘what’ of datasets in relation to the ‘how’ and ‘why’ of AI literacy:

Datasets have a lifecycle of input, usage and interpretation. It is important that at each stage of the lifecycle, the students are supported in how they relate to the data and its interpretation.

(limitations and implications of current datasets for diverse learners,  i.e., [capabilities (physical and neurological)/transdisciplinary (varied skills and competencies)/international (cultural, linguistic, ethnic, religious, racial diversities) and how teachers can utilise culturally responsive learning and teaching methods – (section to be added with further research)]