MEDS Class of 2025

Program Learning Outcome (PLO) #1 Assessment - Core Knowledge

Author

Jamie Montgomery

Published

August 13, 2024

Modified

August 13, 2024

Summary

This PLO assessment was administered to the MEDS class of 2025 both before beginning the program, on August 2, 2024 (response rate = 29/29 students).

The survey consists of 31 questions (28 multiple choice, 3 short free-response) and takes ~30 minutes to complete. Questions 1 - 15 ask respondents to rank how often they use certain data science tools or workflows, or familiarity / comfort levels with particular topics (see Part 1 through Part 4, below). Questions 16 - 31 assess respondents’ familiarity and application of domain-specific knowledge / tools taught during the MEDS program (see Part 5 onward, below). Many of these question types are multi-part and begin with a question phrased as:

  • “How familiar / comfortable are you with X” (rank 1 (never heard of it) > 5 (very familiar))
  • “Have often have you done / implemented Y” (rank 1 (never) > 5 (all the time))

If a respondent chooses a level of 2 or greater, they proceed to the remaining part(s) of the question to be tested on their knowledge / understanding of that topic. If a respondent chooses a level 1, they are skipped to the next question. These questions act as “gates” which prevent respondents who are unfamiliar with a topic from proceeding to and guessing on questions which are meant to test knowledge / understanding.

There are 14 questions which have a correct answer. These questions always follow at least one “gate” question. The results of these questions are presented as a comparison between the percentage of Pre- and Post-MEDS respondents who answered it correctly. It’s important to note that these percentages are calculated based on the total number of students who participated in the assessment (that is 29 for the Pre-MEDS assessment and X for the Post-MEDS assessment), and not the number of students who proceeded past the “gate” question(s).

A perfect score is 14 points. The median score was 4.75.

Individual Questions

Part 1: OS and data/document storage

NOTE: Percentages will not sum to 100%

Part 2: How often do you currently use the following?

Part 3: Workflow satisfaction

Part 4: Rank the following from 1 (strongly disagree) to 5 (strongly agree)

Unusual respondent

Part 5: Stats

All students answered all parts of question 17 in both Pre- and Post-MEDS assessments
  • Pre-MEDS: 28 / 29 student respondents (96.5517241%) chose a familiarity level of 2 or greater, and therefore were directed to answer all parts of Question 17b.

Below is a chunk of code showing a simple linear regression relating the number of pieces of microplastics to the number of days per year with rainfall.

Respondents submitted a wide variety of answers – responses as they were recorded are included in the tables, below:

All X Post-MEDS respondents were directed to answer Question 18b, as compared to 28/29 respondents in the Pre-MEDS PLO Assessment.
  • Pre-MEDS: 28/29 student respondents (96.6%) chose a familiarity level of 2 or greater, and therefore were directed to answer Question 18b. Those who selected a familiarity level of 1 (1 students) were jumped directly to Question 19.

Note

Only students who chose 2 or greater in Question 18a were directed to answer Question 18b. A fully correct answer means choosing exactly the following options: normal, uniform, bimodal, symmetrical.

Part 6: Programming 1

Note
  • Pre-MEDS: 25/29 student respondents (86.2068966%) chose a familiarity level of 2 or greater, and therefore were directed to answer question 19b.

Note
  • Pre-MEDS: 29/29 student respondents (100%) chose a familiarity level of 2 or greater, and therefore were directed to answer question 19c.

The following code (in R) defines a function:

compute_turbine_power <- function(height, flowrate, efficiency, maxheight){
  
  if (height < maxheight) {
    
    power = height * flowrate * efficiency
    
  } else {
    
    power = maxheight * flowrate * efficiency
    
  }
  
  return(power)
  
}

This R code applies this function to data:

flowrate = 2
maxheight = 20
power_turbine_a <- compute_turbine_power(10, flowrate, 0.5, maxheight)

Part 7: Environmental Modeling

All X Post-MEDS respondents were directed to answer Question 20b, as compared to the 16/29 respondents in the Pre-MEDS PLO Assessment.
  • Pre-MEDS: 16/29 student respondents (55.2%) answered “Yes” to Question 20a, and therefore were directed to answer Question 20b. Those who selected answered “No” (13 students) were jumped directly to Question 21.

q20b_yes_post/q20b_total_answers_post Post-MEDS respondents were directed to answer Question 20c, as compared to the 6/16 respondents in the Pre-MEDS PLO Assessment.
  • Pre-MEDS: 6/16 student respondents (37.5%) answered “Yes” to Question 20b, and therefore were directed to answer Question 20c. Those who answered “No” (13 students) were jumped directly to Question 21.

Part 8: Geospatial Analysis & Remote Sensing

All q21a_not1_post/q21a_num_answers_post Post-MEDS respondents were directed to answer Questions 21b & 21c, as compared to the q21a_not1_pre/q21a_num_answers_pre respondents in the Pre-MEDS PLO assessment.
  • Pre-MEDS: 23/29 student respondents (79.3%) chose a familiarity level of 2 or greater, and therefore were directed to answer Questions 21b & 21c. Those who answered “1” (6 students) were jumped directly to Question 22.

  • Post-MEDS: q21a_not1_post/q21a_num_answers_post student respondents (round((q21a_not1_post/q21a_num_answers_post)*100, 1)%) answered “Yes” to Question 21a, and therefore were directed to answer Questions 21b & 21c.

Note

All q22a_not1_post/q22a_num_answers_post Post-MEDS respondents were directed to answer Question 22b, as compared to the q22a_not1_pre/q22a_num_answers_pre respondents in the Pre-MEDS PLO assessment.
  • Pre-MEDS: 14/29 student respondents (48.3%) chose a familiarity level of 2 or greater, and therefore were directed to answer Question 22b. Those who answered “1” (15 students) were jumped directly to Question 23.

  • Post-MEDS: q22a_not1_post/q22a_num_answers_post student respondents (round((q22a_not1_post/q22a_num_answers_post)*100, 1)%) chose a familiarity level of 2 or greater, and therefore were directed to answer Question 22b.

All q23a_not1_post/q23a_num_answers_post Post-MEDS respondents were directed to answer Question 23b, as compared to the q23a_not1_pre/q23a_num_answers_pre respondents in the Pre-MEDS PLO assessment.
  • Pre-MEDS: 20/29 student respondents (69%) chose a familiarity level of 2 or greater, and therefore were directed to answer Question 23b. Those who answered “1” (15 students) were jumped directly to Question 24.

  • Post-MEDS: q23a_not1_post/q23a_num_answers_post student respondents (round((q23a_not1_post/q23a_num_answers_post)*100, 1)%) chose a familiarity level of 2 or greater, and therefore were directed to answer Question 23b.

q24a_not1_post/q24a_num_answers_post Post-MEDS respondents were directed to answer Question 24b, as compared to the 5/29 respondents in the Pre-MEDS PLO Assessment.
  • Pre-MEDS: 5/29 student respondents (17.2%) chose a familiarity level of 2 or greater, and therefore were directed to answer Question 24b. Those who answered “1” (24 students) were jumped directly to Question 25.

  • Post-MEDS: q24a_not1_post/q24a_num_answers_post student respondents (round((q24a_not1_post/q24a_num_answers_post)*100, 1)%) chose a familiarity level of 2 or greater, and therefore were directed to answer Question 24b. Those who answered “1” (q24a_1_post student) were jumped directly to Question 25.

Part 9: Machine Learning

All q25a_not1_post/q25a_num_answers_post Post-MEDS respondents were directed to answer Question 25b, as compared to the q25a_not1_pre/q25a_num_answers_pre respondents in the Pre-MEDS PLO assessment.
  • Pre-MEDS: 22/29 student respondents (75.9%) chose a familiarity level of 2 or greater, and therefore were directed to answer Question 25b. Those who answered “1” (7 students) were jumped directly to Question 26 – these non-responses are missing from the Question 25b plot, below, which is why percentages for the Pre-MEDS assessment do not add up to 100%.

  • Post-MEDS: q25a_not1_post/q25a_num_answers_post student respondents (round((q25a_not1_post/q25a_num_answers_post)*100, 1)%) chose a familiarity level of 2 or greater, and therefore were directed to answer Question 25b.

All q25b_not1_post/q25b_num_answers_post Post-MEDS respondents were directed to answer Question 25c, as compared to the q25b_not1_pre/q25b_num_answers_pre respondents in the Pre-MEDS PLO assessment.
  • Pre-MEDS: 11/29 student respondents (37.9%) chose a familiarity level of 2 or greater, and therefore were directed to answer Question 25b. Those who answered “1” (18 students) were jumped directly to Question 26.

  • Post-MEDS: q25b_not1_post/q25b_num_answers_post student respondents (round((q25b_not1_post/q25b_num_answers_post)*100, 1)%) chose a familiarity level of 2 or greater, and therefore were directed to answer Question 25c.

All q26a_not1_post/q26a_num_answers_post Post-MEDS respondents were directed to answer Question 26b, as compared to the q26a_not1_pre/q26a_num_answers_pre respondents in the Pre-MEDS PLO assessment.
  • Pre-MEDS: 14/29 student respondents (48.3%) chose a familiarity level of 2 or greater, and therefore were directed to answer Question 26b. Those who answered “1” (15 students) were jumped directly to Question 27 – these non-responses are missing from the Question 26b plot, below, which is why percentages for the Pre-MEDS assessment do not add up to 100%.

  • Post-MEDS: q26a_not1_post/q26a_num_answers_post student respondents (round((q26a_not1_post/q26a_num_answers_post)*100, 1)%) chose a familiarity level of 2 or greater, and therefore were directed to answer Question 26b.

All q26b_not1_post/q26b_num_answers_post Post-MEDS respondents were directed to answer Question 26c, as compared to the q26b_not1_pre/q26b_num_answers_pre respondents in the Pre-MEDS PLO assessment.
  • Pre-MEDS: 7/29 student respondents (24.1%) chose a familiarity level of 2 or greater, and therefore were directed to answer Question 26c. Those who answered “1” (7 students) were jumped directly to Question 27 – these non-responses are missing from the Question 26c plot, below, which is why percentages for the Pre-MEDS assessment do not add up to 100%.

  • Post-MEDS: q26b_not1_post/q26b_num_answers_post student respondents (round((q26b_not1_post/q26b_num_answers_post)*100, 1)%) chose a familiarity level of 2 or greater, and therefore were directed to answer Question 26c.

The percentage of respondents who provided a fully correct answer to Queston 26c increased from Pre- to Post-MEDS PLO assessments

Only students who chose 2 or greater in Question 26b were directed to answer Question 26c. A fully correct answer means choosing exactly the following options: My model is overfitting the training set, My model is unlikely to perform well when applied to new data.

Part 10: Environmental Justice

Part 11: Data Viz & Communication

Identify 4 areas for improvement in the following data visualization that shows information about Michigan counties with highest college attendance (Question 30).

Wordcloud of most frequently occurring words used to describe suggested improvements to the above data visualization (Question 30)

Pre-MEDS

Post-MEDS

Responses as they were recorded are included in the tables, below:

Part 12: Programming 2

# define function
def convert_F_to_C(temp_F):
  temp_C = (temp_F-32)*5/9
  return temp_C

# use function
convert_F_to_C(32)


End MEDS Class of 2025 PLO Assessment Report


Return to main page