MEDS Class of 2025

Program Learning Outcome (PLO) #1 Assessment - Core Knowledge

Author

Sam Shanny-Csik & Jamie Montgomery

Published

August 13, 2024

Modified

June 23, 2025

Summary

This PLO assessment was administered to the MEDS class of 2025 both before beginning the program, on August 2, 2024 (response rate = 29/29 students), and again after completing the program, on June 4, 2024 (response rate = 30/30 students).

Discrepancy in total # of students in Pre- and Post-MEDS PLO assessments

One student from the class of 2025 completed MEDS in two years and therefore only took this PLO assessment at the end of the program.

The survey consists of 31 questions (28 multiple choice, 3 short free-response) and takes ~30 minutes to complete. Questions 1 - 15 ask respondents to rank how often they use certain data science tools or workflows, or familiarity / comfort levels with particular topics (see Part 1 through Part 4, below). Questions 16 - 31 assess respondents’ familiarity and application of domain-specific knowledge / tools taught during the MEDS program (see Part 5 onward, below). Many of these question types are multi-part and begin with a question phrased as:

  • “How familiar / comfortable are you with X” (rank 1 (never heard of it) > 5 (very familiar))
  • “Have often have you done / implemented Y” (rank 1 (never) > 5 (all the time))

If a respondent chooses a level of 2 or greater, they proceed to the remaining part(s) of the question to be tested on their knowledge / understanding of that topic. If a respondent chooses a level 1, they are skipped to the next question. These questions act as “gates” which prevent respondents who are unfamiliar with a topic from proceeding to and guessing on questions which are meant to test knowledge / understanding.

There are 14 questions which have a correct answer. These questions always follow at least one “gate” question. The results of these questions are presented as a comparison between the percentage of Pre- and Post-MEDS respondents who answered it correctly. It’s important to note that these percentages are calculated based on the total number of students who participated in the assessment (that is 29 for the Pre-MEDS assessment and 30 for the Post-MEDS assessment), and not the number of students who proceeded past the “gate” question(s).

A perfect score is 14 points. The median score increased by 142.1% from the pre- to post-MEDS PLO assessments.

Individual Questions

Part 1: OS and data/document storage

NOTE: Percentages will not sum to 100%

Part 2: How often do you currently use the following?

Part 3: Workflow satisfaction

Part 4: Rank the following from 1 (strongly disagree) to 5 (strongly agree)

Unusual respondent

One respondent in the Post-MEDS PLO assessment answered 1 (strongly disagree) to all questions 9 - 15.

Part 5: Stats

All Post-MEDS respondents were directed to answer Question 17b, as compared to 28 / 29 respondents in the Pre-MEDS PLO Assessment.
  • Pre-MEDS: 28 / 29 student respondents (96.6%) chose a familiarity level of 2 or greater, and therefore were directed to answer all parts of Question 17b.

  • Post-MEDS: 30 / 30 student respondents (100%) chose a familiarity level of 2 or greater, and therefore were directed to answer all parts of Question 17b.

Below is a chunk of code showing a simple linear regression relating the number of pieces of microplastics to the number of days per year with rainfall.

Respondents submitted a wide variety of answers – responses as they were recorded are included in the tables, below:

All Post-MEDS respondents were directed to answer Question 18b, as compared to 28 / 29 respondents in the Pre-MEDS PLO Assessment.
  • Pre-MEDS: 28 / 29 student respondents (96.6%) chose a familiarity level of 2 or greater, and therefore were directed to answer all parts of Question 18b.

  • Post-MEDS: 30 / 30 student respondents (100%) chose a familiarity level of 2 or greater, and therefore were directed to answer all parts of Question 18b.

The percentage of respondents who provided a fully correct answer to Queston 18b increased from Pre- to Post-MEDS PLO assessments

Only students who chose 2 or greater in Question 18a were directed to answer Question 18b. A fully correct answer means choosing exactly the following options: normal, uniform, bimodal, symmetrical.

Part 6: Programming 1

All Post-MEDS respondents were directed to answer Question 19b, as compared to 25 / 29 respondents in the Pre-MEDS PLO Assessment.
  • Pre-MEDS: 25/29 student respondents (86.2%) chose a familiarity level of 2 or greater, and therefore were directed to answer question 19b.

  • Post-MEDS: 30/30 student respondents (100%) chose a familiarity level of 2 or greater, and therefore were directed to answer question 19b.

All Post-MEDS respondents were directed to answer Question 19c, as compared to 21 / 29 respondents in the Pre-MEDS PLO Assessment.
  • Pre-MEDS: 21/29 student respondents (72.4%) chose a familiarity level of 2 or greater, and therefore were directed to answer question 19c.

  • Post-MEDS: 30/30 student respondents (100%) chose a familiarity level of 2 or greater, and therefore were directed to answer question 19c.

The following code (in R) defines a function:

compute_turbine_power <- function(height, flowrate, efficiency, maxheight){
  
  if (height < maxheight) {
    
    power = height * flowrate * efficiency
    
  } else {
    
    power = maxheight * flowrate * efficiency
    
  }
  
  return(power)
  
}

This R code applies this function to data:

flowrate = 2
maxheight = 20
power_turbine_a <- compute_turbine_power(10, flowrate, 0.5, maxheight)

Part 7: Environmental Modeling

All Post-MEDS respondents were directed to answer Question 20b, as compared to the 16/29 respondents in the Pre-MEDS PLO Assessment.
  • Pre-MEDS: 16/29 student respondents (55.2%) answered “Yes” to Question 20a, and therefore were directed to answer Question 20b. Those who selected answered “No” (13 students) were jumped directly to Question 21.

  • Post-MEDS: 30/30 student respondents (100%) answered “Yes” to Question 20a, and therefore were directed to answer Question 20b.

All Post-MEDS respondents were directed to answer Question 20c, as compared to the 6/16 respondents in the Pre-MEDS PLO Assessment.
  • Pre-MEDS: 6/16 student respondents (37.5%) answered “Yes” to Question 20b, and therefore were directed to answer Question 20c. Those who answered “No” (13 students) were jumped directly to Question 21.

  • Post-MEDS: 30/30 student respondents (100%) answered “Yes” to Question 20b, and therefore were directed to answer Question 20c.

Part 8: Geospatial Analysis & Remote Sensing

29/30 Post-MEDS respondents were directed to answer Questions 21b & 21c, as compared to the 23/29 respondents in the Pre-MEDS PLO assessment.
  • Pre-MEDS: 23/29 student respondents (79.3%) chose a familiarity level of 2 or greater, and therefore were directed to answer Questions 21b & 21c. Those who answered “1” (6 students) were jumped directly to Question 22.

  • Post-MEDS: 29/30 student respondents (96.7%) answered “Yes” to Question 21a, and therefore were directed to answer Questions 21b & 21c. Those who answered “1” (1 students) were jumped directly to Question 22.

Note

All Post-MEDS respondents were directed to answer Question 22b, as compared to the 14/29 respondents in the Pre-MEDS PLO assessment.
  • Pre-MEDS: 14/29 student respondents (48.3%) chose a familiarity level of 2 or greater, and therefore were directed to answer Question 22b. Those who answered “1” (15 students) were jumped directly to Question 23.

  • Post-MEDS: 30/30 student respondents (100%) chose a familiarity level of 2 or greater, and therefore were directed to answer Question 22b.

All Post-MEDS respondents were directed to answer Question 23b, as compared to the 20/29 respondents in the Pre-MEDS PLO assessment.
  • Pre-MEDS: 20/29 student respondents (69%) chose a familiarity level of 2 or greater, and therefore were directed to answer Question 23b. Those who answered “1” (15 students) were jumped directly to Question 24.

  • Post-MEDS: 30/30 student respondents (100%) chose a familiarity level of 2 or greater, and therefore were directed to answer Question 23b.

29/30 Post-MEDS respondents were directed to answer Question 24b, as compared to the 5/29 respondents in the Pre-MEDS PLO Assessment.
  • Pre-MEDS: 5/29 student respondents (17.2%) chose a familiarity level of 2 or greater, and therefore were directed to answer Question 24b. Those who answered “1” (24 students) were jumped directly to Question 25.

  • Post-MEDS: 29/30 student respondents (96.7%) chose a familiarity level of 2 or greater, and therefore were directed to answer Question 24b. Those who answered “1” (1 student) were jumped directly to Question 25.

Part 9: Machine Learning

All Post-MEDS respondents were directed to answer Question 25b, as compared to the 22/29 respondents in the Pre-MEDS PLO assessment.
  • Pre-MEDS: 22/29 student respondents (75.9%) chose a familiarity level of 2 or greater, and therefore were directed to answer Question 25b. Those who answered “1” (7 students) were jumped directly to Question 26 – these non-responses are missing from the Question 25b plot, below, which is why percentages for the Pre-MEDS assessment do not add up to 100%.

  • Post-MEDS: 30/30 student respondents (100%) chose a familiarity level of 2 or greater, and therefore were directed to answer Question 25b.

All Post-MEDS respondents were directed to answer Question 25c, as compared to the 11/29 respondents in the Pre-MEDS PLO assessment.
  • Pre-MEDS: 11/29 student respondents (37.9%) chose a familiarity level of 2 or greater, and therefore were directed to answer Question 25b. Those who answered “1” (18 students) were jumped directly to Question 26.

  • Post-MEDS: 30/30 student respondents (100%) chose a familiarity level of 2 or greater, and therefore were directed to answer Question 25c.

All Post-MEDS respondents were directed to answer Question 26b, as compared to the 14/29 respondents in the Pre-MEDS PLO assessment.
  • Pre-MEDS: 14/29 student respondents (48.3%) chose a familiarity level of 2 or greater, and therefore were directed to answer Question 26b. Those who answered “1” (15 students) were jumped directly to Question 27 – these non-responses are missing from the Question 26b plot, below, which is why percentages for the Pre-MEDS assessment do not add up to 100%.

  • Post-MEDS: 30/30 student respondents (100%) chose a familiarity level of 2 or greater, and therefore were directed to answer Question 26b.

All Post-MEDS respondents were directed to answer Question 26c, as compared to the 7/29 respondents in the Pre-MEDS PLO assessment.
  • Pre-MEDS: 7/29 student respondents (24.1%) chose a familiarity level of 2 or greater, and therefore were directed to answer Question 26c. Those who answered “1” (7 students) were jumped directly to Question 27 – these non-responses are missing from the Question 26c plot, below, which is why percentages for the Pre-MEDS assessment do not add up to 100%.

  • Post-MEDS: 30/30 student respondents (100%) chose a familiarity level of 2 or greater, and therefore were directed to answer Question 26c.

The percentage of respondents who provided a fully correct answer to Queston 26c increased from Pre- to Post-MEDS PLO assessments

Only students who chose 2 or greater in Question 26b were directed to answer Question 26c. A fully correct answer means choosing exactly the following options: My model is overfitting the training set, My model is unlikely to perform well when applied to new data.

Part 10: Environmental Justice

Part 11: Data Viz & Communication

Identify 4 areas for improvement in the following data visualization that shows information about Michigan counties with highest college attendance (Question 30).

Wordcloud of most frequently occurring words used to describe suggested improvements to the above data visualization (Question 30)

Pre-MEDS

Post-MEDS

Responses as they were recorded are included in the tables, below:

Part 12: Programming 2

# define function
def convert_F_to_C(temp_F):
  temp_C = (temp_F-32)*5/9
  return temp_C

# use function
convert_F_to_C(32)


End MEDS Class of 2025 PLO Assessment Report


Return to main page