fish_indent_lexer: explicitly encode/decode bytes over pipe

Universal newlines behaves differently between Python 2.7 and 3.x,
leading to problems when running Sphinx with Python 2.7.

fish_indent always uses \n, so there's no need to use universal newline
detection.

This also allows full UTF-8 in documentation sources.

Closes #5808.
This commit is contained in:
David Adam 2019-04-12 23:07:41 +08:00
parent a8030c020b
commit 2ca1bc433f

View file

@ -89,11 +89,11 @@ def tokenize_fish_command(code, offset):
["fish_indent", "--pygments"],
stdin=subprocess.PIPE,
stdout=subprocess.PIPE,
universal_newlines=True,
universal_newlines=False,
)
stdout, _ = proc.communicate(code)
stdout, _ = proc.communicate(code.encode('utf-8'))
result = []
for line in stdout.splitlines():
for line in stdout.decode('utf-8').splitlines():
start, end, role = line.split(",")
start, end = int(start), int(end)
value = code[start:end]