The ethics of preventing third-party net filtering

I haven’t posted anything research-related in a while because I’ve been on a project that I’m not supposed to talk about till it’s done, and it’s not done yet. I can say, though, that it’s about ways to get around country-scale filtration of the Internet. I’m writing it up now, starting with the threat model, as you do:

Alice Arishat wishes to publish things for Brutus to read. Cato does not approve of what Arishat has to say, and seeks to prevent her from publishing anything.

Most online discussion of censorship starts from the premise that Cato is automatically in the wrong here. That’s one of the cypherpunk premises that underpin most discussion of theoretical Internet security. I want to play devil’s advocate today, though, and explore circumstances where we might choose to support Cato. In the offline world, we trade off free speech against all sorts of other values every day:

Cato is a government. Arishat is criticizing its policies.
Core political speech receives consistent, strong protection from US courts, even when large groups wish it didn’t (e.g. flag desecration, neofascist marches).

Cato is a policeman on duty in a public place. Arishat is documenting his actions.
This is also mostly agreed to be protected by the First Amendment, but the police don’t like it and they still try to stop people.

Cato holds the copyright on a Great Work of Literature. Arishat has written a parody, homage, fanfic, or critique with extensive quotations, without Cato’s approval.
US law is broadly sympathetic to Cato; in particular, the DMCA takedown mechanism makes it very easy for him to get Arishat’s works pulled offline. Arishat may be entitled to claim the fair use exception to copyright, and actual court cases in this area tend to be on the side of parodists. However, copyright lawsuits are expensive, and Cato is likely to have a lot more money than Arishat does.

Arishat and Cato’s business partnership fell apart. Arishat is now trying to ruin Cato by publishing lies about him.
This is defamation, a common-law tort; Cato can file a lawsuit and the courts will force Arishat to publish a retraction, take down the original lies, and/or pay damages. But there are a lot of restrictions; most importantly, Cato has to prove that what Arishat said was false (the precise legal standard varies by jurisdiction and whether or not Cato is a public figure), and Arishat can argue that Cato is [abusing the courts to suppress debate of an issue of public concern][]. This kind of lawsuit is not nearly as expensive as copyright lawsuits, and it’s more likely that Arishat and Cato have similar amounts of money. Also, if Arishat posted the lies in a public online forum, Cato can’t sue the forum.

Cato is a private citizen. Arishat has posted embarrassing pictures of him online, and then offered to take them back down—for a fee.
This is blackmail, which is a crime (not just a tort); if the police can be bothered to investigate, Arishat is going to jail.

Cato runs an internet forum devoted to gardening. Arishat is trying to stir up some lulz by posting disturbing cartoon images, categorist jokes, and/or off-topic logorrhea on random threads.
As long as Cato is a private citizen, it is perfectly legal for Cato to delete everything Arishat posts, on sight; this is considered the same as throwing a drunk asshole out of your house party before they ruin it for everyone else. Further, all evidence from the last 20+ years of online fora is that if Cato doesn’t do something to get Arishat to stop, it will become impossible to talk about gardening on his forum. However, in any case that is not perfectly clear-cut, and some that are, Cato is likely to be subject to endless, vicious criticism of his decisions.

When Cato is not a private citizen, his ability to keep the trolls out may be limited lest he use that as an excuse to suppress legitimate arguments. Similarly, US jurisdictions do not agree whether shopping mall owners have to permit people to do anything but shop in their space.


We can also write the threat model from Brutus’s perspective:

Brutus wishes to read things Arishat has published. Cato wishes to prevent Brutus from reading anything he considers inappropriate.

and produce another list of difficult scenarios:

Brutus is a child, Cato is his father.
Most people will agree that most children are not ready to experience the full variety of material that adults are expected to handle. Most people will also agree that no two children are the same and parents are in the best position to judge what their own children are ready for. However, you can make a strong case that Brutus should be taught certain things whether or not Cato approves, such as reading, arithmetic, and the theory of evolution.

Brutus wishes to watch videos of people having sex. Cato thinks that will turn Brutus into a sexual predator.
Cato is wrong; access to pornography appears to reduce the incidence of rape, contra an awful lot of fulmination on the subject.

Brutus wants to know whether Cato’s restaurant is any good. Cato would rather he not find any negative reviews.
We understand where Cato is coming from but we don’t see why we should help, unless the negative reviews are being posted by disgruntled ex-employee Arishat, which may be a case of defamation (see above).

Cato is the present government of a country that engaged in crimes against humanity quite some time ago. They are so ashamed of this that they wish to erase all public legacy of the ruling ideology of the time; therefore they criminalize the use of all its symbols and the sale of related memorabilia.
Again, we understand where Cato is coming from, but we suspect the tactic is counterproductive. Actual instances of this particular Cato all have neo-ideological movements. Note that in at least one case, one such country has tried to make these laws extend to actions committed on foreign soil (but visible to its nationals).

Brutus wants to know whether or not he should vote for Cato in the next election. Cato doesn’t want him to find any news articles about his alleged cocaine habit.
Now we’re back to core political speech.


We have a whole bunch of difficult scenarios here, and deciding which scenario we’re in requires human judgement. A computer (short of a fully fledged human-equivalent AI) cannot, even in principle, tell whether or not the string of bits that Arishat posted online is protected free speech. This is part of why the cypherpunks take the position that Cato is always wrong: that’s a position you can enforce with code. However, I would argue that this is too inflexible and leads to undesirable consequences. Via entirely social means, it is already well-nigh impossible to make something completely disappear once it has been put online, and we can easily find cases where that was a bad thing.

So here’s a proposal: I conjecture that the following statements of principle are an appropriate synthesis of Arishat, Brutus, and Cato’s legitimate interests:

  • Arishat should be able to publish online while concealing her offline identity versus anything short of legal process.
  • Arishat should be able to publish whatever she likes in cyberspace she controls without first getting Cato’s approval.
  • Brutus should be able to access Arishat’s publication space, and Cato should be completely unable to tell whether or not he has done this.
  • Cato should be able to control what appears in his own space, however, if he permits any third-party material to appear, his editing or removal of that material should be subject to audit by the general public.
  • Cato should have some recourse after the fact if Arishat posts something in her own space that is genuinely harmful to his interests, but this should involve a heavyweight, public, transparent process with a disinterested arbiter, such as a lawsuit.

Discuss.

Responses to “The ethics of preventing third-party net filtering”

  1. Robert O'Callahan

    Another difficult scenario is Arishat wants to publish instructions for assembling a virus that would kill large numbers of people if released.

    1. Zack Weinberg

      I tend to think the instructions for constructing something very dangerous scenario is overblown. There is no equivalent in microbiology for the turnkey exploits that we’re used to in computer security. I know people who do DIY genetic engineering in their kitchens; it requires a nontrivial investment in equipment, a bunch of background knowledge to even know what the instructions mean, and a fair bit of practice to make anything interesting happen. Right now, I may be being overoptimistic about the goodwill of microbiologists worldwide toward humanity in general, but I think that anyone who wanted to make a deadly, contagious virus probably couldn’t do anything with the instructions even if they had them, and anyone who has the skill and the equipment to do something with the instructions probably doesn’t want to.

      Now, if we ever get Diamond Age style matter compilers, this will become a much more serious issue—but at that point I would advocate that it is hopeless to try to police the use of the devices, and we should work instead toward a society where no one is so disgruntled that they want to take out a city.

  2. Anonymous

    I would argue that more information always works better than less information. Cato has no right to stop Arishat under any circumstances. However, Cato has the right to talk too, and attempt to supply information of his own.

    Regarding your proposed set of rules, they look good with a couple of notable bugs:

    First, versus anything short of legal process does not work; your threat model needs to include legal process as something potentially hostile. Any loophole you leave open for legal process will get abused.

    Second, Cato should have complete and total control over his own publication space. While in principle I agree that viewers of that space ought to have some knowledge of the changes over time, including any removed content, that remains up to Cato. Services like the Internet Archive can provide useful third-party verification, but Cato himself should have no requirement to make the changes to his site auditable.

    Finally, Cato has a recourse if Arishat publishes something harmful about Cato: Cato can publish his own information countering what Arishat published. Let the truth and evidence win. If any process exists to compel Arishat to change what she published, even a noisy public process, that process will get abused.

    1. Robert O'Callahan

      I don’t understand how fundamentalist cypherpunks can maintain that more information always works better than less information. Wouldn’t that mean that publishing people’s passwords on the Internet must be a good thing?

      1. cyberpunk

        Qba’g cbfg qhzo fghss. Gunaxf!

        MODERATOR HAT ON: This level of rudeness is unacceptable. Your comment has been rot13ed. If you continue to attack other posters rather than contributing to the discussion you will be blocked. –zw

        1. Zack Weinberg

          Ooh, an actual troll in my post about, among other things, how to deal with trolls! … The line between legitimate moderator actions and suppressing criticism deserves an entire post of its own, though. For right now, I’ll just leave the above as a demonstration of what I think is appropriate given the technology available to me.

    2. Zack Weinberg

      It is not an accident that practically all of my Cato-vs-Arishat scenarios involved the courts: since no later than the invention of equity the court system has been how this class of argument gets resolved, and for the most part—concerns of expense and political abuse aside—it’s pretty damn good at it! Refusing to consider the possibility that legal process might be the best available tool for the job is a case of NIH syndrome. That said, I sympathize with the argument that the legal system of any given country has to be considered potentially hostile—but I don’t think it will work to design the system so that the courts get no say. If nothing else, that is the kind of attitude that gets your whole business sanctioned.

      I do NOT sympathize with the notion that Cato’s only recourse should be to publish a rebuttal. That just plain doesn’t work. Arishat may have a much bigger megaphone, or Cato may be in a position where anything he could say would only make the situation worse.

      I also don’t (for purposes of this discussion, anyway) agree with the notion that Cato should have total, unauditable control over his own publication space. That leads directly to the situation where Cato deletes every comment that disagrees with him even a little; which can be a big deal even when Cato is a private citizen (see baleeting in the context of online flamewars) and is definitely unacceptable when Cato is, say, a government running a forum that is supposed to be for debate of its policies, but could easily turn into an echo chamber of yes-men.

      By the way, I just edited your post to correct a typo (Arishat is a she). I could have modified your text to make you sound like a caricature of yourself, and Wordpress wouldn’t show any evidence that it hadn’t always been that way. I didn’t, but I could have. Are you really happy with that?

  3. Justin L.
    Arishat should be able to publish online while concealing her offline identity versus anything short of legal process.

    What jurisdiction does the legal process happen in? I may trust the US courts, but what if Cato controls the courts in my country? Certainly that’s the case in most (all?) countries with internet filtration systems.

    1. Tony Mechelynck

      And as a follow-up to what Justin said: if Arishat, Brutus and Cato are from three different countries, which court(s) shall judge the case? And (considering the laws and customs about extradition, where most countries are reluctant to extradite their nationals), if a single case is tried in Arisha’s country in favour of Arisha, in Brutus’s country in favour of Brutus, and in Cato’s country in favour of Cato, what will happen then?

      BTW, I may trust the Belgian courts (i.e. the courts of my country), but I’m far from certain that I would trust a US court in a case between me and, hm, let’s say Microsoft or Google.

      1. Zack Weinberg

        This is a really good question, and I don’t have a good answer for it. In a better world, where people all lived under jurisdictions that they were personally comfortable with, it would be logical to say that the jurisdiction of the writer applies. But that’s very much not the world we have, and the Arishat wants to write something that Arishat’s own government disapproves of scenario is, alas, going to remain relevant to these discussions going forward.

        The hosting provider for this site, under limited circumstances and with great caution, will provide anonymous hosting for people not resident in the USA where the content to be hosted is critical of [the author’s] government in a manner that would be Constitutionally protected speech in the United States. (Further elaboration.) I don’t know, at all, how this plays out in practice; but I find it an interesting case study.

  4. Gervase Markham

    Your synthesis doesn’t seem to do anything with the Brutus is a child, Cato is his father scenario. Is that intentional?

    I think that these scenarios get far more interesting when you consider the relationship between Brutus and Cato than when you are thinking about Arisha and Cato.

    1. Zack Weinberg

      I’m the opposite; I find the Arishat scenarios more interesting, and I can’t at present think of any scenarios where I would support Cato over Brutus, which is why the only principle in my list for Cato vs Brutus online is basically that Brutus wins.

      Even in the parent/child case, the furthest I would ever go with my own (hypothetical) child is That book will make more sense to you when you’re older. If you really want to read it now, you can try, and I’ll try to explain anything that doesn’t make sense, but I may not be able to make it make sense. Or perhaps No, you may not watch that scary movie right now, because you have school tomorrow, and the last time you watched a scary movie you didn’t get any sleep the night afterward. Maybe on the weekend. I have a good deal of faith in my hypothetical children’s ability to handle stuff, I guess.

      I’d be interested to hear Cato/Brutus scenarios that you think are interesting, though. It is quite possible that I am missing something important.

      1. Gervase Markham

        I am fascinated that you would prevent your child watching a particular movie for practical reasons (school tomorrow) but not for the reason that you thought the content was in itself inappropriate or immoral or corrupting. My reading of your comment is that you barely admit the existence of the category. It seems to me that this can be for only one of three reasons; either a) you don’t believe that what a child watches, reads or hears has any effect on the formation of their moral character, or b) you have no interest in your child forming a particular moral character, but are content to let it be formed by the action of whatever inputs he or she happens to come into contact with; or c) you think there’s no such thing as moral character (and therefore, it would seem to me, deny the existence of morality in general).

        Which is it? Or is there a d) I’ve missed?

        1. Zack Weinberg

          I believe that the set of inputs that are intrinsically harmful to the development of moral character is either nonexistent or so small as to be negligible.

          What harms said development, in my view, is mostly restricting one’s reading/watching/hearing to a limited range of perspectives, or uncritical consumption of any particular item; and the critical eye is itself best developed by exposure to a wide range of content. Therefore, I’m primarily concerned with the breadth of what my hypothetical child takes in, rather than any particular thing it may or may not contain, and I mean to make a point of discussing everything with them.

          Could you give some examples of material that you would consider intrinsically harmful to children’s moral development?