assignments:assignment3
Differences
This shows you the differences between two versions of the page.
| Both sides previous revision Previous revision Next revision | Previous revision Next revision Both sides next revision | ||
|
assignments:assignment3 [2016/09/15 14:49] asa [Submission] |
assignments:assignment3 [2016/09/19 10:09] asa |
||
|---|---|---|---|
| Line 3: | Line 3: | ||
| ====== Assignment 3 ====== | ====== Assignment 3 ====== | ||
| - | **Due:** 10/2 at 11pm. | + | **Due:** 10/3 at 11:59pm. |
| ===== Preliminaries ===== | ===== Preliminaries ===== | ||
| Line 11: | Line 11: | ||
| The wine data is composed of two datasets - one for white wines, and one for reds. In this assignment perform all your analyses on just the red wine data. | The wine data is composed of two datasets - one for white wines, and one for reds. In this assignment perform all your analyses on just the red wine data. | ||
| - | The features for the wine dataset are not standardized, so make sure you do this, especially since we are going to consider the magnitude of the weight vector (recall that standardization entails subtracting the mean and then dividing by the standard deviation for each feature; you can use the [[http://docs.scipy.org/doc/numpy/reference/routines.statistics.html | Numpy statistics module]] to perform the required calculations). | + | The features for the wine dataset are not standardized, so make sure you do this, especially since we are going to consider the magnitude of the weight vector (recall that standardization entails subtracting the mean and then dividing by the standard deviation for each feature; you can use the [[http://docs.scipy.org/doc/numpy/reference/routines.statistics.html | Numpy statistics module]] to perform the required calculations). |
| ==== Part 1 ==== | ==== Part 1 ==== | ||
| - | Implement ridge regression keeping the same API you used in implementing the classifiers in assignment 2, and functions for computing the following measures of error: | + | Implement ridge regression in a class called RidgeRegression that implements the classifier API, i.e. ``fit`` and ``predict`` methods with the same signature as the Perceptron class you implemented in the previous assignment. Also implement functions for computing the following measures of error: |
| * The Root Mean Square Error (RMSE). | * The Root Mean Square Error (RMSE). | ||
| Line 30: | Line 31: | ||
| With the code you just implemented, your next task is to explore the dependence of error on the value of the regularization parameter, $\lambda$. | With the code you just implemented, your next task is to explore the dependence of error on the value of the regularization parameter, $\lambda$. | ||
| In what follows set aside 30% of the data as a test-set, and compute the in-sample error, and the test-set error as a function of the parameter $\lambda$ on the red wine data. Choose the values of $\lambda$ on a logarithmic scale with values 0.01, 0.1, 1, 10, 100, 1000 and plot the RMSE only. | In what follows set aside 30% of the data as a test-set, and compute the in-sample error, and the test-set error as a function of the parameter $\lambda$ on the red wine data. Choose the values of $\lambda$ on a logarithmic scale with values 0.01, 0.1, 1, 10, 100, 1000 and plot the RMSE only. | ||
| - | Repeat the same experiment where instead of using all the training data, choose 20 random training examples. | + | Repeat the same experiment where instead of using all the training data, choose 20 random examples out of the training set, and train your model using those 20 examples. |
| Now answer the following: | Now answer the following: | ||
| Line 58: | Line 59: | ||
| Next, perform the following experiment: | Next, perform the following experiment: | ||
| Incrementally remove the feature with the lowest absolute value of the weight vector and retrain the ridge regression classifier. | Incrementally remove the feature with the lowest absolute value of the weight vector and retrain the ridge regression classifier. | ||
| - | Plot RMSE as a function of the number of features that remain on the test set which you have set aside. | + | Plot RMSE as a function of the number of features that remain on the test set which you have set aside and comment on the results. |
| ===== Submission ===== | ===== Submission ===== | ||
| Line 68: | Line 69: | ||
| </code> | </code> | ||
| should generate all the tables/plots used in your report. | should generate all the tables/plots used in your report. | ||
| + | |||
| Line 73: | Line 75: | ||
| ===== Grading ===== | ===== Grading ===== | ||
| - | Here is what the grade sheet will look like for this assignment. A few general guidelines for this and future assignments in the course: | + | A few general guidelines for this and future assignments in the course: |
| - | * Always provide a description of the method you used to produce a given result in sufficient detail such that the reader can reproduce your results on the basis of the description (UNLESS the method has been provided in class or is there in the book). Your code needs to be provided in sufficient detail so we can make sure that your implementation is correct. The saying that "the devil is in the details" holds true for machine learning, and is sometimes the makes the difference between correct and incorrect results. If your code is more than a few lines, you can include it as an appendix to your report, or submit it as a separate file. Make sure your code is readable! | + | * Your answers should be concise and to the point. |
| - | * You can provide results in the form of tables, figures or text - whatever form is most appropriate for a given problem. | + | * You need to use LaTex to write the report. |
| + | * The report is well structured, the writing is clear, with good grammar and correct spelling; good formatting of math, code, figures and captions (every figure and table needs to have a caption that explains what is being shown). | ||
| + | * Whenever you use information from the web or published papers, a reference should be provided. Failure to do so is considered plagiarism. | ||
| + | |||
| + | We will take off points if these guidelines are not followed. | ||
| + | |||
| + | * Always provide a description of the method you used to produce a given result in sufficient detail such that the reader can reproduce your results on the basis of the description. You can use a few lines of python code or pseudo-code. | ||
| + | * You can provide results in the form of tables, figures or text - whatever form is most appropriate for a given problem. There are no rules about how much space each answer should take. BUT we will take off points if we have to wade through a lot of redundant data. | ||
| * In any machine learning paper there is a discussion of the results. There is a similar expectation from your assignments that you reason about your results. For example, for the learning curve problem, what can you say on the basis of the observed learning curve? | * In any machine learning paper there is a discussion of the results. There is a similar expectation from your assignments that you reason about your results. For example, for the learning curve problem, what can you say on the basis of the observed learning curve? | ||
| - | * Write succinct answers. We will take off points for rambling answers that are not to the point, and and similarly, if we have to wade through a lot of data/results that are not to the point. | ||
| <code> | <code> | ||
| - | Grading sheet for assignment 2 | + | Grading sheet for assignment 3 |
| Part 1: 50 points. | Part 1: 50 points. | ||
| - | (20 points): Plots of MAD and RMSE as a function of lambda are generated correctly. | + | (15 points): Ridge regression is correctly implemented. |
| - | (20 points): REC curves are generated correctly | + | (15 points): Plots of RMSE as a function of lambda are generated correctly. |
| - | ( 5 points): discussion of REC curves | + | (20 points): Discussion of the results |
| - | ( 5 points): Discussion of the MAD and RMSE plots and comparison of results to the published ones. | + | |
| - | Part 2: 40 points. | + | Part 2: 25 points. |
| - | (30 points): Weight vector analysis | + | (15 points): REC curves are generated correctly |
| - | (10 points): Comparison to the published results | + | (10 points): discussion of REC curves |
| + | |||
| + | Part 3: 25 points. | ||
| + | (20 points): Weight vector analysis | ||
| + | ( 5 points): Comparison to the published results | ||
| - | Report structure, grammar and spelling: 10 points | ||
| - | (10 points): Heading and subheading structure easy to follow and clearly divides report into logical sections. | ||
| - | Code, math, figure captions, and all other aspects of the report are well-written and formatted. | ||
| - | Grammar, spelling, and punctuation. | ||
| </code> | </code> | ||
| + | |||
assignments/assignment3.txt · Last modified: 2016/09/20 09:34 by asa
Page Tools
Except where otherwise noted, content on this wiki is licensed under the following license: CC Attribution-Share Alike 4.0 International
