Working with Segmented String Relative Rankings

Gepubliceerd door admin op

Indra Overland and Javlon Juraev proposed a method and algorithm to work effectively and efficiently with segmented string relative rankings (SSRRs) for producing rankings based on expert surveys. The method is described in this paper and a good example of its use is described in this paper (references at the bottom). Using SSRRs has several advantages, but the most useful advantages are:

  • The experts that participate do not have to grade according to a format developed by the researchers.
  • The experts do not have to rank all the items, just the ones they are knowledgeable on.
  • Since this is an automated algorithm there is less room for human error.
  • The algorithm works well with relatively few participants and rankings.
  • The algorithm signals when there is too little data for certain items to rank.

Although, as this is an algorithm that works with php-code on a command line, using it can be quite intimidating. Therefore, I have written this short manual to get you started. However, first want to mention that this algorithm is the amazing work of Indra Overland and Javlon Juraev, I am just an enthusiastic user that wants to spread a little knowledge on how to use the algorithm.

STEP ONE: prepare your data

You need to prepare your data very carefully, as the algorithm needs a very specifically formatted file. There is an example file in the original paper (see screenshot below) and there is an example file in this link.

  1. Your file needs to have no more or less than three columns. In the first column, there there should be the unit, in the second the expert and in the third the ranking. This means that there will be multiple rows per expert!
  2. Make sure you have no column headers (for example describing the content of the column) or row headers (for example row numbers).
  3. The file should be formatted as a .CSV file.
  4. Use UNIX / Mac OS line endings (LF).
  5. Use a comma (,) as list separator.
  6. Use Unicode (UTF-8) encoding.
Example of data formatting

STEP TWO: prepare the code

  1. Find an app to run php, preferably with as little other functionality as possible to keep it simple. On Mac, PHP runner is a good option. On iOS DraftCode is very useful.
  2. Download the PHP-file from this link or contact the developers of the algorithm for the most recent version.
  3. Open the PHP-file in the app.

STEP THREE: run the algorithm

  1. In the php-code, on line 11, you have to insert the path to the data-file you have prepared in step one. This is behind the “fopen” command (see screenshot below). If you are not sure how to do this, there are plenty of tutorials for example on YouTube.
  2. Check that the output is in HTML (otherwise it will be hard to read).
  3. Run the code
  4. You should now have results!
Example of data-path

References:

Categorieën: Kennis