Differences

This shows you the differences between two versions of the page.

--- code:model_selection [2015/10/05 13:25]
asa created
+++ code:model_selection [2015/10/05 13:49]
asa
@@ Line 20: / Line 20: @@
 </code>
+The simplest form of model evaluation uses a validation/test set:
+<code python>
+In [9]: X_train, X_test, y_train, y_test = cross_validation.train_test_split(X, y, test_size=0.4, random_state=0)
+In [10]: classifier = svm.SVC(kernel='linear', C=1).fit(X_train, y_train)
+In [11]: classifier.score(X_test, y_test)
+Out[11]: 0.7592592592592593
+</code>
+Next, let'd perform cross-validation:
+<code python>
+In [12]: scores = cross_validation.cross_val_score(classifier, X, y, cv=5, scoring='accuracy')
+In [13]:
+In [13]: scores = cross_validation.cross_val_score(classifier, X, y, cv=5, scoring='roc_auc')
+In [14]: # you can also obtain the predictions by cross-validation and then compute the accuracy:
+In [15]: y_predict = cross_validation.cross_val_predict(classifier, X, y, cv=5)
+In [16]: metrics.accuracy_score(y, y_predict)
+Out[16]: 0.83703703703703702
+</code>

CS545 fall 2016