๐Ÿ‘ To work for society, data scientists need a Hippocratic oath with teeth

Liked To work for society, data scientists need a hippocratic oath with teeth by Tom Upchurch (WIRED UK)

The first question is, are the algorithms that we deploy going to improve the human processes that they are replacing? Far too often we have algorithms that are thrown in with the assumptions that theyโ€™re going to work perfectly, because after all theyโ€™re algorithms, but they actually end up working much worse than the system that theyโ€™re replacing. For example in Australia they implemented an algorithm that sent a bunch of ridiculously threatening letters to people saying that they had defrauded the Australian Government. Thatโ€™s a great example where they actually just never tested it to make sure it worked.

The second question is to ask, for whom is the algorithm failing? We need to be asking, โ€œDoes it fail more often for women than for men? Does it fail more often for minorities than for whites? Does it fail more often for old people than for young people?โ€ Every single class should get a question and an answer. The big example I have for this one is the facial recognition software that the MIT Media Lab found worked muchbetter for white men than black women. That is a no-brainer test that every single facial recognition software company should have done and itโ€™s embarrassing that they didnโ€™t do it.

The third category of question is simply, is this working for society? Are we tracking the mistakes of the system? Are we inputting these mistakes back into the algorithm so that itโ€™ll work better? Is it causing some other third unintended consequence? Is it destroying democracy? Is it making people worse off?

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.

To respond on your own website, enter the URL of your response which should contain a link to this post's permalink URL. Your response will then appear (possibly after moderation) on this page. Want to update or remove your response? Update or delete your post and re-enter your post's URL again. (Learn More)