CS253 HW4: Unicode!                

Description                

Write a C++ program called uni.cc that behaves as HW3, except that it processes UTF-8 characters, as described in lecture.                 

Properties file                

The properties file can now be enormous. Here is an example: ~cs253/pub/UnicodeData.txt. Note that the properties file now has a slightly different format than the previous, ASCII-only, file.

• Fields in each line are delimited by semicolons
• The first field is four, five, or six hexadecimal digits
• The property is the third field

As before, the properties file need not be in any particular order, and “holes” may exist.                 

The -all option is still valid, and refers to all the properties found in the properties file.                 

Arguments                

Arguments are in this order:                 

• options, which start with a - (a dash, minus, or hyphen)
• properties file
• one or more data files

How do you tell which arguments are options?                

They’re the leading arguments that start with a dash.                 

How do I know where the options end?                

When an argument is encountered without a dash, or you run out of arguments.                 

What if a later argument starts with a dash?                

What if it does? It’s not an option. All the options are together, at the beginning. Anything after that is not an option.                 

Sample Runs                

Here are several sample runs. “%” is my prompt:                 

    % echo -e 'a 5 \U3a9 \U4dca \U1f42e' >data
    % cat data
    a 5 Ω ䷊ 🐮
    % ./a.out ~cs253/pub/UnicodeData.txt data
    % ./a.out -Lu -Ll ~cs253/pub/UnicodeData.txt data
    Ll: 1
    Lu: 1
    % ./a.out -all ~cs253/pub/UnicodeData.txt data
    Cc: 1
    Cf: 0
    Co: 0
    Cs: 0
    Ll: 1
    Lm: 0
    Lo: 0
    Lt: 0
    Lu: 1
    Mc: 0
    Me: 0
    Mn: 0
    Nd: 1
    Nl: 0
    No: 0
    Pc: 0
    Pd: 0
    Pe: 0
    Pf: 0
    Pi: 0
    Po: 0
    Ps: 0
    Sc: 0
    Sk: 0
    Sm: 0
    So: 2
    Zl: 0
    Zp: 0
    Zs: 4
    % ./a.out -all ~cs253/pub/UnicodeData.txt ~cs253/pub/hamlet.txt
    Cc: 10412
    Cf: 0
    Co: 0
    Cs: 0
    Ll: 119683
    Lm: 0
    Lo: 0
    Lt: 0
    Lu: 16350
    Mc: 0
    Me: 0
    Mn: 0
    Nd: 0
    Nl: 0
    No: 0
    Pc: 0
    Pd: 422
    Pe: 261
    Pf: 0
    Pi: 0
    Po: 7418
    Ps: 261
    Sc: 0
    Sk: 0
    Sm: 48
    So: 0
    Zl: 0
    Zp: 0
    Zs: 27713

Requirements                

The requirements are the same as those for HW3, plus:                 

• You may not assume that the input files contain valid UTF-8 encoded characters. If an improperly-formatted byte (for example, the first byte is 0xFF) is encountered, display an appropriate message to standard error, including the program name, filename, and two-digit hexadecimal representation of the byte, and then stop.
• Do not treat Byte order marks specially.
• You may assume that the Unicode characters described by the properties file are in the range U+0000–U+10FFFF.
• We may run your program with the properties files shown above, or with other files with different names and different contents.
• You may not use C-style dynamic memory via malloc, calloc, realloc, free, and the like.
  • Use new/delete/new[]/delete[], if you must do your own memory allocation.
  • Containers are easier!
• Do it yourself:
  • You may not use u16_string(cpp), u32_string(cpp), wstring, wcout, wchar_t, etc.
  • You may not use any of the C++ wide-character facilities.
  • You may not use an external program via popen, system, or the like.
  • You have to do any UTF-8 character decoding/encoding yourself.

Ask                

If you have any questions about the requirements, ask. In the real world, your programming tasks will almost always be vague and incompletely specified. Same here.                 

How to submit your homework:                

Use web checkin, or Linux checkin:                 

    ~cs253/bin/checkin HW4 uni.cc

How to receive negative points:                

Turn in someone else’s work.