MEDS Class of 2024

Program Learning Outcome (PLO) #1 Assessment - Core Knowledge

Author

Sam Csik

Published

August 4, 2023

Modified

July 15, 2024

Summary

This PLO assessment was administered to the MEDS class of 2024 both before beginning the program, on August 4, 2023 (response rate = 31/31 students), and again after completing the program, on June 5, 2024 (response rate = 29/31 students).

The survey consists of 31 questions (28 multiple choice, 3 short free-response) and takes ~30 minutes to complete. Questions 1 - 15 ask respondents to rank how often they use certain data science tools or workflows, or familiarity / comfort levels with particular topics (see Part 1 through Part 4, below). Questions 16 - 31 assess respondents’ familiarity and application of domain-specific knowledge / tools taught during the MEDS program (see Part 5 onward, below). Many of these question types are multi-part and begin with a question phrased as:

  • “How familiar / comfortable are you with X” (rank 1 (never heard of it) > 5 (very familiar))
  • “Have often have you done / implemented Y” (rank 1 (never) > 5 (all the time))

If a respondent chooses a level of 2 or greater, they proceed to the remaining part(s) of the question to be tested on their knowledge / understanding of that topic. If a respondent chooses a level 1, they are skipped to the next question. These questions act as “gates” which prevent respondents who are unfamiliar with a topic from proceeding to and guessing on questions which are meant to test knowledge / understanding.

There are 14 questions which have a correct answer. These questions always follow at least one “gate” question. The results of these questions are presented as a comparison between the percentage of Pre- and Post-MEDS respondents who answered it correctly. It’s important to note that these percentages are calculated based on the total number of students who participated in the assessment (that is 31 for the Pre-MEDS assessment and 29 for the Post-MEDS assessment), and not the number of students who proceeded past the “gate” question(s).

A perfect score is 14 points. The median score increased by 109.1% from the pre- to post-MEDS PLO assessments.

Individual Questions

Part 1: OS and data/document storage

NOTE: Percentages will not sum to 100%

Part 2: How often do you currently use the following?

Part 3: Workflow satisfaction

Part 4: Rank the following from 1 (strongly disagree) to 5 (strongly agree)

Unusual respondent

One respondent in the Post-MEDS PLO assessment answered 1 (strongly disagree) to all questions 10 - 15.

Part 5: Stats

All students answered all parts of question 17 in both Pre- and Post-MEDS assessments
  • Pre-MEDS: 31 / 31 student respondents (100%) chose a familiarity level of 2 or greater, and therefore were directed to answer all parts of Question 17b.

  • Post-MEDS: 29 / 29 student respondents (100%) chose a familiarity level of 2 or greater, and therefore were directed to answer all parts of Question 17b.

Below is a chunk of code showing a simple linear regression relating the number of pieces of microplastics to the number of days per year with rainfall.

Respondents submitted a wide variety of answers – responses as they were recorded are included in the tables, below:

All 29 Post-MEDS respondents were directed to answer Question 18b, as compared to 27/31 respondents in the Pre-MEDS PLO Assessment.
  • Pre-MEDS: 27/31 student respondents (87.1%) chose a familiarity level of 2 or greater, and therefore were directed to answer Question 18b. Those who selected a familiarity level of 1 (4 students) were jumped directly to Question 19.

  • Post-MEDS: 29/29 student respondents (100%) chose a familiarity level of 2 or greater, and therefore were directed to answer Question 18b.

The percentage of respondents who provided a fully correct answer to Queston 18b increased from Pre- to Post-MEDS PLO assessments

Only students who chose 2 or greater in Question 18a were directed to answer Question 18b. A fully correct answer means choosing exactly the following options: normal, uniform, bimodal, symmetrical.

Part 6: Programming 1

All students advanced to question 19b in both Pre- and Post-MEDS assessments
  • Pre-MEDS: 31/31 student respondents (100%) chose a familiarity level of 2 or greater, and therefore were directed to answer question 19b.

  • Post-MEDS: 29/29 student respondents (100%) chose a familiarity level of 2 or greater, and therefore were directed to answer question 19b.

All students advanced to question 19c in both Pre- and Post-MEDS assessments
  • Pre-MEDS: 31/31 student respondents (100%) chose a familiarity level of 2 or greater, and therefore were directed to answer question 19c.

  • Post-MEDS: 29/29 student respondents (100%) chose a familiarity level of 2 or greater, and therefore were directed to answer question 19c.

The following code (in R) defines a function:

compute_turbine_power <- function(height, flowrate, efficiency, maxheight){
  
  if (height < maxheight) {
    
    power = height * flowrate * efficiency
    
  } else {
    
    power = maxheight * flowrate * efficiency
    
  }
  
  return(power)
  
}

This R code applies this function to data:

flowrate = 2
maxheight = 20
power_turbine_a <- compute_turbine_power(10, flowrate, 0.5, maxheight)

Part 7: Environmental Modeling

All 29 Post-MEDS respondents were directed to answer Question 20b, as compared to the 16/31 respondents in the Pre-MEDS PLO Assessment.
  • Pre-MEDS: 16/31 student respondents (51.6%) answered “Yes” to Question 20a, and therefore were directed to answer Question 20b. Those who selected answered “No” (15 students) were jumped directly to Question 21.

  • Post-MEDS: 29/29 student respondents (100%) answered “Yes” to Question 20a, and therefore were directed to answer Question 20b.

28/29 Post-MEDS respondents were directed to answer Question 20c, as compared to the 7/16 respondents in the Pre-MEDS PLO Assessment.
  • Pre-MEDS: 7/16 student respondents (43.8%) answered “Yes” to Question 20b, and therefore were directed to answer Question 20c. Those who answered “No” (15 students) were jumped directly to Question 21.

  • Post-MEDS: 28/29 student respondents (96.6%) answered “Yes” to Question 20b, and therefore were directed to answer Question 20c. Those who answered “No” (1 student) were jumped directly to Question 21.

Part 8: Geospatial Analysis & Remote Sensing

All 29/29 Post-MEDS respondents were directed to answer Questions 21b & 21c, as compared to the 24/31 respondents in the Pre-MEDS PLO assessment.
  • Pre-MEDS: 24/31 student respondents (77.4%) chose a familiarity level of 2 or greater, and therefore were directed to answer Questions 21b & 21c. Those who answered “1” (7 students) were jumped directly to Question 22.

  • Post-MEDS: 29/29 student respondents (100%) answered “Yes” to Question 21a, and therefore were directed to answer Questions 21b & 21c.

The percentage of respondents who provided a fully correct answer to Question 21b increased from Pre- to Post-MEDS PLO assessments.

All 29/29 Post-MEDS respondents were directed to answer Question 22b, as compared to the 16/31 respondents in the Pre-MEDS PLO assessment.
  • Pre-MEDS: 16/31 student respondents (51.6%) chose a familiarity level of 2 or greater, and therefore were directed to answer Question 22b. Those who answered “1” (15 students) were jumped directly to Question 23.

  • Post-MEDS: 29/29 student respondents (100%) chose a familiarity level of 2 or greater, and therefore were directed to answer Question 22b.

All 29/29 Post-MEDS respondents were directed to answer Question 23b, as compared to the 15/31 respondents in the Pre-MEDS PLO assessment.
  • Pre-MEDS: 15/31 student respondents (48.4%) chose a familiarity level of 2 or greater, and therefore were directed to answer Question 23b. Those who answered “1” (15 students) were jumped directly to Question 24.

  • Post-MEDS: 29/29 student respondents (100%) chose a familiarity level of 2 or greater, and therefore were directed to answer Question 23b.

28/29 Post-MEDS respondents were directed to answer Question 24b, as compared to the 6/31 respondents in the Pre-MEDS PLO Assessment.
  • Pre-MEDS: 6/31 student respondents (19.4%) chose a familiarity level of 2 or greater, and therefore were directed to answer Question 24b. Those who answered “1” (25 students) were jumped directly to Question 25.

  • Post-MEDS: 28/29 student respondents (96.6%) chose a familiarity level of 2 or greater, and therefore were directed to answer Question 24b. Those who answered “1” (1 student) were jumped directly to Question 25.

Part 9: Machine Learning

All 29/29 Post-MEDS respondents were directed to answer Question 25b, as compared to the 20/31 respondents in the Pre-MEDS PLO assessment.
  • Pre-MEDS: 20/31 student respondents (64.5%) chose a familiarity level of 2 or greater, and therefore were directed to answer Question 25b. Those who answered “1” (11 students) were jumped directly to Question 26 – these non-responses are missing from the Question 25b plot, below, which is why percentages for the Pre-MEDS assessment do not add up to 100%.

  • Post-MEDS: 29/29 student respondents (100%) chose a familiarity level of 2 or greater, and therefore were directed to answer Question 25b.

All 29/29 Post-MEDS respondents were directed to answer Question 25c, as compared to the 21/31 respondents in the Pre-MEDS PLO assessment.
  • Pre-MEDS: 21/31 student respondents (67.7%) chose a familiarity level of 2 or greater, and therefore were directed to answer Question 25b. Those who answered “1” (10 students) were jumped directly to Question 26.

  • Post-MEDS: 29/29 student respondents (100%) chose a familiarity level of 2 or greater, and therefore were directed to answer Question 25c.

All 29/29 Post-MEDS respondents were directed to answer Question 26b, as compared to the 14/31 respondents in the Pre-MEDS PLO assessment.
  • Pre-MEDS: 14/31 student respondents (45.2%) chose a familiarity level of 2 or greater, and therefore were directed to answer Question 26b. Those who answered “1” (17 students) were jumped directly to Question 27 – these non-responses are missing from the Question 26b plot, below, which is why percentages for the Pre-MEDS assessment do not add up to 100%.

  • Post-MEDS: 29/29 student respondents (100%) chose a familiarity level of 2 or greater, and therefore were directed to answer Question 26b.

All 29/29 Post-MEDS respondents were directed to answer Question 26c, as compared to the 9/31 respondents in the Pre-MEDS PLO assessment.
  • Pre-MEDS: 9/31 student respondents (29%) chose a familiarity level of 2 or greater, and therefore were directed to answer Question 26c. Those who answered “1” (5 students) were jumped directly to Question 27 – these non-responses are missing from the Question 26c plot, below, which is why percentages for the Pre-MEDS assessment do not add up to 100%.

  • Post-MEDS: 29/29 student respondents (100%) chose a familiarity level of 2 or greater, and therefore were directed to answer Question 26c.

The percentage of respondents who provided a fully correct answer to Queston 26c increased from Pre- to Post-MEDS PLO assessments

Only students who chose 2 or greater in Question 26b were directed to answer Question 26c. A fully correct answer means choosing exactly the following options: My model is overfitting the training set, My model is unlikely to perform well when applied to new data.

Part 10: Environmental Justice

Part 11: Data Viz & Communication

Identify 4 areas for improvement in the following data visualization that shows information about Michigan counties with highest college attendance (Question 30).

Wordcloud of most frequently occurring words used to describe suggested improvements to the above data visualization (Question 30)

Pre-MEDS

Post-MEDS

Responses as they were recorded are included in the tables, below:

Part 12: Programming 2

# define function
def convert_F_to_C(temp_F):
  temp_C = (temp_F-32)*5/9
  return temp_C

# use function
convert_F_to_C(32)


End MEDS Class of 2024 PLO Assessment Report


Return to main page