Will Jardee

Will Jardee

Smell the roses and do the theory.

A Lesson in Rubrics

3 minute read

Published:

I grade for a graduate level course in Machine Leasrning. As most people who teach in universities, I have never had any formal training. All my training has come from person experience, what I have been able to pull out of other teachers (themselves not being trained), and research. With this base, the following is a recreation of the “rubric” that I was able to come up with for the assignment:

criteriapointsearnedcomments
Grammar4  
Structure4  
Hypothesis5  
Experimental Design20  
Conclusion/Analysis20  

Later editions of this rubric would breakdown blocks into smaller components to try to help explain what goes into each:

  • Experimental Design: ## / 20
    • Methods: ## / 5
    • Validity: ## / 5
    • CV: ## / 2
    • All datasets: ## / 4
    • Communication: ## / 4

As you might expect, this did little to effectively teach the students what what missing and how to improve. If you are familiar with rubrics at all, you should easily point out that this is not a rubric in the slightest. It is indeed a weighted checklist. One of the biggest weaknesses of this style is that an explanation of the points deducted has to be explained from comments left in the assignment/on the rubric or through direct conversations with students. This means that the students are not effectively getting the information that they need, especially if they don’t reach out about deducted points, and I have to spend a lot of time commenting PDFs or typing out emails. There has got to be a better way to do this! (Hint: anyone who knows anything about grading/teaching knows that there are rubrics)

So, on to writing rubrics I went. If you don’t know how a rubric should look, it is typically in the form of:

criteria4321
Some criteriaWhat defines a 4/4 on this sectionA little less than 4/4, but better than a 2/4What defines 2/4What defines a 1/4

Some rubrics have 3 tiers, some have 5. What if you don’t want to grade on a discrete scale? Allow decimals to be used and sort them into the larger of the two boxes. Student’s can read this as a weighted average of the two blocks (a 3.4 will have 40% of the 4 and 60% of the 3). What if you don’t want to limit the scores of the criteria to such low values? Using the numbers as a fraction of the total score instead. So, 3/4 will be 75% of the total possible points. So, if you will to let Experimental Design to have 20 points, and the student achieves a 3/4, then they would get (20 * 3/4 = 15) points. My course uses Canvas, and Canvas’s rubric functionality allows you to automatically scale values by changing the “out of” score.

So, what are the benefits? TODO: Add research that defends the use of rubrics.

You May Also Enjoy

Directions I Give Students When Writing Papers

14 minute read

Published:

As we gear up to submit the first project, I have collected common issues student’s have had in previous years. These concepts are reflected in current machine learning research. To get a better grasp on how these expectations all come together to make a complete project, consider reading some recent research papers; however, note that many of these papers have issues and do not deliver on all our expectations. Each project can be seen as a lab you would run in physics or chemistry, and the final paper you submit is the lab report. This works for a computer science course because machine learning is a noisy, application based field, one where empirical tests align well with the problems and the attempted solution (this is very similar to have computer systems and applied algorithms research). Your version of setting up petri dishes, doing titrations, or shining a laser in the right place is setting up a well motivated, cross-validated experiment on datasets. When running your tests, writing your paper, recording your demo, and commenting your code, keep this in mind. What you provide must:

  1. meet the writing conventions of the field,
  2. be well formulated, motivated, and defendable,
  3. be reproducible, assuming hardware and time is available, and
  4. draw conclusions and state deficiencies of the experiments. How these concepts will be graded can be seen in the rubric attached to each assignment. Please make sure to read it completely, and reach out if you have questions. I also have documents of each rubric that can be cast into an accessible font (OpenDyslexic, Atkinson Hyperlegible, etc.) if you need. Please email me if you need that service.

How I Keep Track of my Research

12 minute read

Published:

Through my undergrad and in my first year of grad school, I had no clue how to do background research. I’m talking finding papers, keep track of what I have read (hopefully with some simple notes or the pdf’s too), and how to find gaps or future directions to investigate. While I am still new to the space and am likely rediscovering the wheel, I struggled finding anyone who laid out their “wheel design” plainly. So, let’s do that here.

A Short List of Some Historical AI Models

10 minute read

Published:

As I am putting together more work for the AI exhibit, it is becoming clear just how many pivotal models exist. As the list continues to grow, it only makes sense to make an itemized list of them. See below for what I have collected and the appropriate links I have gathered to understand them.