The objective of this course is to provide students with a working knowledge of the state of research and practice on software product and process evaluation and improvement. Students completing this course should be able to evaluate both software artifacts and software processes. Software product evaluation includes the evaluation of external characteristics such as usability and functionality, as well as internal characteristics such as design structure, testability and maintainability. Software process evaluation addresses the problem of understanding the software development process, assessing the quality of the process, and developing plans for process improvement. The use of quantitative methods in software engineering will be stressed. The course will also cover experimental methods as they are applied to evaluating software engineering methods.

Software testing is a major component of software product evaluation; testing consumes a large portion of most software development budgets. We will focus on testing object-oriented systems. Issues of interest include testing theory, the implications of design choices on testing, and practical issues such as selecting and creating test data, testing tools, and the design of test support software.

The topics covered in this course are listed below:

• Software Testing
  • Build/Inspection/Test Processes
  • Input Space Partitioning
  • Test Requirements and Graphs
  • Logic Coverage
  • Fault Models
  • Testing OO Software
  • Test Harness Design
  • Mutation Testing
  • Overview of topics for research (regression testing, UI testing, AR/VR testing, test input generation, fault localization, automatic program repair, fuzz testing)
• Measurement
  • Foundations of Measurement
  • Goal-Based Measurement Approach
  • Goals and Process Evaluation
  • Empirical Studies
  • Data Analysis
  • Metrics for Decision Support
  • Causal Models
  • Internal/External Quality

CS 414 (and by transitivity CS 314), or consent of instructor. Students are expected to know the basics of software engineering as taught in an undergraduate course using a text such as Pressman, Software Engineering: A Practitioners Approach, or Somerville, Software Engineering. Students should have experience in designing and debugging object-oriented software, and have a basic understanding of object modeling notations such as the UML. Students are expected to have the basic background of the undergraduate core of computer science, which includes discrete math (sets, graphs, first-order predicate calculus), data structures, operating systems, and basic probability and statistics.

You must have a Colorado State University eIdentity (eID), before you can be installed into the CS514 Canvas system. The CS514 Canvas page will have all of the course notes, assignments, discussions, and exams, so it is important for you to be installed on this system. Visit the eIdentity and eServices web page to get your eID. You will not be able to take part in the course until you have an eID.

All queries to the instructor and TA should be sent by email to cs514@cs.colostate.edu, which will ensure that both of them get it. You can also post your questions on the discussion board, but only in the General Questions group. Queries posted on any other discussion group will not be monitored.

The first day of class is Wednesday, January 23, 2019. Be ready to start then. Go to the Progress page to view the weekly schedule. The on-campus class meets on Mondays, Wednesdays, and Fridays. We will post video recordings of the on-campus lectures for the benefit of the distance students.

• Required Texts:

  1. N. Fenton and J. Bieman. Software Metrics: A Rigorous and Practical Approach 3nd Edition. CRC Press, 2014. Boston. Int. Thompson Computer Press. ISBN: 1439838224, or 978-1439838228.
  2. Paul Ammann and Jeff Offutt, Introduction to Software Testing, Cambridge University Press, Cambridge, UK. Either Edition 1 (ISBN 0-52188-038-6, 2008) or Edition 2 (ISBN: 9781107172012, 2016). The instructor will refer to the relevant page numbers for each lecture.

• Other material:
  • Video recordings of lectures will be available via Canvas.
  • Powerpoint slides will be available on Canvas under Modules→Slides.
  • Papers from journals and conference proceedings on Canvas.

Here are the formally graded elements of the course and associated weighting:

Final letter grades will be based on the relative distribution of total scores and not on any preset numerical grade.

Assignments need to be turned in electronically via Canvas by 11:59 pm on the due date. All the assignments will be individual. Some will involve coding and others will involve solving problems and writing (typing) the answers in a document.

We will be glad to re-grade the same assignment submission if you feel that there was a mistake in grading. Contact us within 4 calendar days for a re-grade. Work will be re-graded in its entirety, and may result in an increase, decrease, or no change in the grade. Note that, once we have graded an assignment, we will not allow you to re-do and re-submit it for grading.

Students must actively participate in the discussion forums on Canvas to get a grade in discussion assignments. While specific instructions will be given for each discussion assignment, in general, we expect each student to make an initial posting for each assignment, followed by responses to other students' postings. Merely having one sentence that says, for example, "I agree with everyone", is not enough to get a grade.

For the on-campus section, class participation is important and will be determined in part by taking attendance, asking and answering questions in the lecture, and participation in class discussions.

Quizzes will be administered via Canvas and auto-graded. You will get three attempts for each quiz, and the highest score will be considered for the final grade.

There will be one midterm exam and one final exam. Details regarding the exams will be provided later. Tentatively, the midterm will happen during the week of March 4-10. The final exam will happen during finals week. The final exam won't be comprehensive. It will include the material that was not tested on the midterm. Most likely both will be take home exams.

There will be a term project that involves (1) writing a proposal (5%), writing a term paper (15%), and preparing and presenting a poster (5%). Students should identify topics by the second week of March and submit a draft proposal (2%) for feedback (actual day TBD). The final proposal is due after the spring break (3%). The term paper is due the week before finals week (actual day TBD). For on-campus students, the poster presentation will happen sometime during that week using printed posters in the 3rd floor lounge or in the lecture itself. Logistics will be worked out later. Distance students will submit (1) a Poster in PDF format and (2) a 5-minute video displaying the poster on a computer screen accompanied by audio of them explaining the poster.

• Late submissions are not allowed for class participation activities, discussion participation, and quizzes.

• Late submissions may be allowed only with prior permission on a case-by-case basis for assignments and projects. If you cannot finish the work by the deadline, contact the instructor as soon as possible. Extensions will be granted on a case-by-case basis and are more likely when permission is sought in advance, for reasons which are unexpected and beyond your control, and which involve only a short extension. The instructors reserve the right to assign a score penalty to the late work, depending on the circumstances and degree of lateness. Late penalties for assignments and projects are as follows:

  • Up to 12 hours late: 10% penalty
  • 12 to 24 hours late: 25% penalty
  • 24 to 48 hours late: 50% penalty
  • More than 48 hours late: No credit

Note that the percentages refer to the total points available for the deliverable, not the points that you have received. That is, if an assignment is worth 100 points, and you get 76 and are 7 hours late, you will receive 66 points.

• Exams may not be taken or submitted after the due date. If you miss an exam, you receive a score of zero.

All students are expected to conduct themselves professionally. We (the instructors and GTAs) assume you are familiar with the policies in the policies and resources page and in particular, the student information guide and the department conduct code. Additionally, you are computing professionals. You should be familiar with the code of conduct for the primary professional society, ACM. You can read the ACM Code of Conduct.

This course will adhere to the CSU Academic Integrity Policy as found in the Student Conduct Code. At a minimum, violations will result in a grading penalty in this course and a report to the Office of Conflict Resolution and Student Conduct Services.

Students not already familiar with the CSU Honor Pledge should review this clear and simple pledge and always adhere to it. Academic dishonesty will be dealt with severely. The first instance of cheating will result in negative credit. The second instance will result in a failing grade and other penalties dictated by departmental and university policies.

We work to maintain an environment supportive of learning in the classroom and laboratory. Towards that end, we require that you be courteous to and respectful of your fellow participants (i.e., classmates, instructors, GTAs and any tutors). In particular:

• Please turn off the ring on your cell phone. If you are expecting an emergency call, sit near the door and slide out discretely to take it.
• In class use of electronic devices in general, and laptops specifically, is permitted as a courtesy so that you may better participate and learn. If at any time the instructor judges that an electronic device is becoming a distraction the student may be asked to to turn it off and put it away.

We require you to follow the guidelines listed below for postings on Canvas:

• Be professional.
• Be courteous.
• Do not post any questions about your grade or any grading issues. Such questions are private communication and must be emailed.
• Do not post any opinions regarding your graded assignment, or any aspect of the course. You will have a chance to submit anonymous reviews at the end of the semester. Email feedback to the instructor regarding the course is acceptable and welcome.
• The instructor reserves the right to delete postings that do not promote a conducive learning experience for students in the class.