Move cursor on mouse click via kitty's OSC 133 click_events=1

When the user clicks somewhere in the prompt, kitty asks the shell
to move the cursor there (since there is not much else to do).

This is currently implemented by sending an array of
forward-char-passive commands.  This has problems, for example it
is really slow on large command lines (probably because we repaint
everytime).

Implement kitty's `click_events=1` flag to set the
position directly.  To convert from terminal-coordinates
to fish-coordinates, query [CSI 6 n Report Cursor
Position](https://invisible-island.net/xterm/ctlseqs/ctlseqs.html)
and use it to compute the left prompt's terminal-coordinates (which
are (0, 0) in fish-coordinates).

Unfortunately this doesn't work correctly while the terminal is
scrolled.  This is probably because the cursor position is wrong
if off-screen.  To fix that we could probably record the cursor
position while not scrolled, but it doesn't seem terribly important
(the existing implementation also doesn't get it right).

Also add parsing for some unused mouse events; I'll probably remove
them again unless we can use them now.

Future work:

1. Knowledge of the position of the prompt will
   also enable us to make [ctrl-l scroll instead of erasing]
   (https://codeberg.org/dnkl/foot/wiki#how-do-i-make-ctrl-l-scroll-the-content-instead-of-erasing-it)
   (as of wiki commit b57489e298f95d037fdf34da00ea60a5e8eafd6d) which
   might be a better default.

2. We still turn off mouse reporting.  If we turned it on, it
   would be harder to select text in the terminal itself (not fish).
   This would typically mean that mouse-drag will alter fish's
   selection and shift+mouse-drag or alt+mouse-drag can be used.

   To improve this, we could try to synchronize the selection:
   if parts of the fish commandline are selected in the terminal's
   selection, copy that to fish's selection and vice versa.

   Or maybe there is an intuitive criteria, like: whenever we receive
   a mouse event outside fish, turn off mouse reporting, and turn
   it back on whenver we receive new keyboard input.  One problem
   is that we lose one event (though we could send it back to the
   terminal). Another problem is we would turn it back on too late
   in some scenarios.

CHANGELOG.rst

@@ -16,6 +16,7 @@ Interactive improvements
 New or improved bindings
 ^^^^^^^^^^^^^^^^^^^^^^^^
 - :kbd:`ctrl-z` (undo) after executing a command will restore the previous cursor position instead of placing the cursor at the end of the command line.
+- The OSC 133 prompt marking feature has learned about kitty's ``click_events=1`` flag, which allows moving fish's cursor by clicking.
 
 Completions
 ^^^^^^^^^^^

src/input.rs

@@ -6,7 +6,10 @@ use crate::flog::FLOG;
 use crate::input_common::{
     CharEvent, CharInputStyle, InputData, InputEventQueuer, ReadlineCmd, R_END_INPUT_FUNCTIONS,
 };
-use crate::key::{self, canonicalize_raw_escapes, ctrl, Key, Modifiers};
+use crate::key::ViewportPosition;
+use crate::key::{
+    self, canonicalize_raw_escapes, ctrl, Key, Modifiers, MouseButton, MouseEvent, MouseEventType,
+};
 use crate::proc::job_reap;
 use crate::reader::{
     reader_reading_interrupted, reader_reset_interrupted, reader_schedule_prompt_repaint, Reader,
@@ -458,6 +461,28 @@ impl<'a> InputEventQueuer for Reader<'a> {
             escape(&str2wcstring(&buffer))
         )));
     }
+
+    fn handle_mouse_event(&mut self, event: MouseEvent) {
+        FLOG!(reader, "Mouse event", event);
+        if !matches!(event.r#type, MouseEventType::Click(MouseButton::Left)) {
+            return;
+        }
+        if self.pending_mouse_left.is_some() {
+            FLOG!(
+                error,
+                "Received mouse event while still waiting for previous position"
+            );
+            return;
+        }
+        self.pending_mouse_left = Some(event.position);
+        self.screen.write_bytes(b"\x1b[6n");
+    }
+    fn has_pending_mouse_left(&mut self) -> bool {
+        self.pending_mouse_left.is_some()
+    }
+    fn on_mouse_left(&mut self, cursor: ViewportPosition) {
+        self.mouse_left(cursor);
+    }
 }
 
 /// A struct which allows accumulating input events, or returns them to the queue.

src/input_common.rs

@@ -11,7 +11,7 @@ use crate::fork_exec::flog_safe::FLOG_SAFE;
 use crate::global_safety::RelaxedAtomicBool;
 use crate::key::{
     self, alt, canonicalize_control_char, canonicalize_keyed_control_char, function_key, shift,
-    Key, Modifiers,
+    Key, Modifiers, MouseButton, MouseEvent, MouseEventType, ViewportPosition,
 };
 use crate::reader::{reader_current_data, reader_test_and_clear_interrupted};
 use crate::threads::{iothread_port, is_main_thread};
@@ -902,22 +902,72 @@ pub trait InputEventQueuer {
             b'H' => masked_key(key::Home, None), // PC/xterm style
             b'M' | b'm' => {
                 self.disable_mouse_tracking();
+                // Generic X10 or modified VT200 sequence, or extended (SGR/1006) mouse
+                // reporting mode, with semicolon-separated parameters for button code, Px,
+                // and Py, ending with 'M' for button press or 'm' for button release.
                 let sgr = private_mode == Some(b'<');
                 if !sgr && c == b'm' {
                     return None;
                 }
-                // Extended (SGR/1006) mouse reporting mode, with semicolon-separated parameters
+                let button = if sgr {
-                // for button code, Px, and Py, ending with 'M' for button press or 'm' for
+                    params[0][0]
-                // button release.
+                } else {
-                if sgr {
+                    u32::from(next_char(self)) - 32
-                    return None;
+                };
-                }
+                let x = usize::try_from(
-                // Generic X10 or modified VT200 sequence. It doesn't matter which, they're both 6
+                    if sgr {
-                // chars (although in mode 1005, the characters may be unicode and not necessarily
+                        params[1][0]
-                // just one byte long) reporting the button that was clicked and its location.
+                    } else {
-                let _ = next_char(self);
+                        u32::from(next_char(self)) - 32
-                let _ = next_char(self);
+                    } - 1,
-                let _ = next_char(self);
+                )
+                .unwrap();
+                let y = usize::try_from(
+                    if sgr {
+                        params[2][0]
+                    } else {
+                        u32::from(next_char(self)) - 32
+                    } - 1,
+                )
+                .unwrap();
+                let position = ViewportPosition { x, y };
+                let modifiers = parse_mask((button >> 2) & 0x07);
+                let code = button & 0x43;
+                let mouse_event = match code {
+                    0..=2 => {
+                        let button = [MouseButton::Left, MouseButton::Right, MouseButton::Middle]
+                            [usize::try_from(code).unwrap()];
+                        MouseEvent {
+                            modifiers,
+                            r#type: if c == b'm' {
+                                MouseEventType::Release(button)
+                            } else {
+                                MouseEventType::Click(button)
+                            },
+                            position,
+                        }
+                    }
+                    3 => {
+                        // Insert button release handling here.
+                        return None;
+                    }
+                    64 => MouseEvent {
+                        modifiers,
+                        r#type: MouseEventType::ScrollUp,
+                        position,
+                    },
+                    65 => MouseEvent {
+                        modifiers,
+                        r#type: MouseEventType::ScrollUp,
+                        position,
+                    },
+                    _ => MouseEvent {
+                        modifiers,
+                        r#type: MouseEventType::Nop,
+                        position,
+                    },
+                };
+                self.handle_mouse_event(mouse_event);
                 return None;
             }
             b't' => {
@@ -937,7 +987,16 @@ pub trait InputEventQueuer {
             }
             b'P' => masked_key(function_key(1), None),
             b'Q' => masked_key(function_key(2), None),
-            b'R' => masked_key(function_key(3), None),
+            b'R' => {
+                if self.has_pending_mouse_left() {
+                    let y = usize::try_from(params[0][0] - 1).unwrap();
+                    let x = usize::try_from(params[1][0] - 1).unwrap();
+                    self.on_mouse_left(ViewportPosition { x, y });
+                    return None;
+                } else {
+                    masked_key(function_key(3), None)
+                }
+            }
             b'S' => masked_key(function_key(4), None),
             b'~' => match params[0][0] {
                 1 => masked_key(key::Home, None), // VT220/tmux style
@@ -1272,6 +1331,12 @@ pub trait InputEventQueuer {
         }
     }
 
+    fn handle_mouse_event(&mut self, _event: MouseEvent) {}
+    fn has_pending_mouse_left(&mut self) -> bool {
+        false
+    }
+    fn on_mouse_left(&mut self, _cursor: ViewportPosition) {}
+
     /// Override point for when we are about to (potentially) block in select(). The default does
     /// nothing.
     fn prepare_to_select(&mut self) {}

src/key.rs

@@ -3,6 +3,7 @@ use libc::VERASE;
 use crate::{
     common::{escape_string, EscapeFlags, EscapeStringStyle},
     fallback::fish_wcwidth,
+    flog::FloggableDebug,
     reader::TERMINAL_MODE_ON_STARTUP,
     wchar::{decode_byte_from_char, prelude::*},
     wutil::{fish_is_pua, fish_wcstoi},
@@ -77,6 +78,38 @@ impl Modifiers {
     }
 }
 
+#[derive(Clone, Copy, Debug, Eq, PartialEq)]
+pub enum MouseButton {
+    Left,
+    Right,
+    Middle,
+}
+
+#[derive(Clone, Copy, Debug, Eq, PartialEq)]
+pub enum MouseEventType {
+    Click(MouseButton),
+    Release(MouseButton),
+    ScrollUp,
+    ScrollDown,
+    Nop,
+}
+
+/// Position in terminal coordinates, i.e. not starting from the prompt
+#[derive(Clone, Copy, Debug, Default, Eq, PartialEq)]
+pub struct ViewportPosition {
+    pub x: usize,
+    pub y: usize,
+}
+impl FloggableDebug for ViewportPosition {}
+
+#[derive(Clone, Copy, Debug, Eq, PartialEq)]
+pub struct MouseEvent {
+    pub modifiers: Modifiers,
+    pub r#type: MouseEventType,
+    pub position: ViewportPosition,
+}
+impl FloggableDebug for MouseEvent {}
+
 #[derive(Clone, Copy, Debug, Eq, PartialEq)]
 pub struct Key {
     pub modifiers: Modifiers,

src/pager.rs

@@ -252,7 +252,10 @@ impl Pager {
         assert!(stop_row >= start_row);
         assert!(stop_row <= row_count);
         assert!(stop_row - start_row <= term_height);
+        // This always printed at the end of the command line.
+        let offset_in_cmdline = usize::MAX;
         self.completion_print(
+            offset_in_cmdline,
             cols,
             &width_by_column,
             start_row,
@@ -297,6 +300,7 @@ impl Pager {
                 HighlightRole::pager_progress,
             );
             print_max(
+                offset_in_cmdline,
                 &progress_text,
                 spec,
                 term_width,
@@ -325,6 +329,7 @@ impl Pager {
 
         let mut search_field_remaining = term_width - 1;
         search_field_remaining -= print_max(
+            offset_in_cmdline,
             wgettext!(SEARCH_FIELD_PROMPT),
             HighlightSpec::new(),
             search_field_remaining,
@@ -332,6 +337,7 @@ impl Pager {
             search_field,
         );
         search_field_remaining -= print_max(
+            offset_in_cmdline,
             &search_field_text,
             underline,
             search_field_remaining,
@@ -401,6 +407,7 @@ impl Pager {
     /// \param lst The list of completions to print
     fn completion_print(
         &self,
+        offset_in_cmdline: usize,
         cols: usize,
         width_by_column: &[usize; PAGER_MAX_COLS],
         row_start: usize,
@@ -429,12 +436,18 @@ impl Pager {
                 let is_selected = Some(idx) == effective_selected_idx;
 
                 // Print this completion on its own "line".
-                let mut line =
+                let mut line = self.completion_print_item(
-                    self.completion_print_item(prefix, el, col_width, row % 2 != 0, is_selected);
+                    offset_in_cmdline,
+                    prefix,
+                    el,
+                    col_width,
+                    row % 2 != 0,
+                    is_selected,
+                );
 
                 // If there's more to come, append two spaces.
                 if col + 1 < cols {
-                    line.append_str(PAGER_SPACER_STRING, HighlightSpec::new());
+                    line.append_str(PAGER_SPACER_STRING, HighlightSpec::new(), offset_in_cmdline);
                 }
 
                 // Append this to the real line.
@@ -449,6 +462,7 @@ impl Pager {
     /// Print the specified item using at the specified amount of space.
     fn completion_print_item(
         &self,
+        offset_in_cmdline: usize,
         prefix: &wstr,
         c: &PagerComp,
         width: usize,
@@ -510,6 +524,7 @@ impl Pager {
         for (i, comp) in c.comp.iter().enumerate() {
             if i > 0 {
                 comp_remaining -= print_max(
+                    offset_in_cmdline,
                     PAGER_SPACER_STRING,
                     bg,
                     comp_remaining,
@@ -519,6 +534,7 @@ impl Pager {
             }
 
             comp_remaining -= print_max(
+                offset_in_cmdline,
                 prefix,
                 prefix_col,
                 comp_remaining,
@@ -526,6 +542,7 @@ impl Pager {
                 &mut line_data,
             );
             comp_remaining -= print_max_impl(
+                offset_in_cmdline,
                 comp,
                 |i| {
                     if c.colors.is_empty() {
@@ -546,12 +563,13 @@ impl Pager {
         let mut desc_remaining = width - comp_width + comp_remaining;
         if c.desc_width > 0 && desc_remaining > 4 {
             // always have at least two spaces to separate completion and description
-            desc_remaining -= print_max(L!("  "), bg, 2, false, &mut line_data);
+            desc_remaining -= print_max(offset_in_cmdline, L!("  "), bg, 2, false, &mut line_data);
 
             // right-justify the description by adding spaces
             // the 2 here refers to the parenthesis below
             while desc_remaining > c.desc_width + 2 {
-                desc_remaining -= print_max(L!(" "), bg, 1, false, &mut line_data);
+                desc_remaining -=
+                    print_max(offset_in_cmdline, L!(" "), bg, 1, false, &mut line_data);
             }
 
             assert!(desc_remaining >= 2);
@@ -563,14 +581,35 @@ impl Pager {
                 },
                 bg_role,
             );
-            desc_remaining -= print_max(L!("("), paren_col, 1, false, &mut line_data);
+            desc_remaining -= print_max(
-            desc_remaining -=
+                offset_in_cmdline,
-                print_max(&c.desc, desc_col, desc_remaining - 1, false, &mut line_data);
+                L!("("),
-            desc_remaining -= print_max(L!(")"), paren_col, 1, false, &mut line_data);
+                paren_col,
+                1,
+                false,
+                &mut line_data,
+            );
+            desc_remaining -= print_max(
+                offset_in_cmdline,
+                &c.desc,
+                desc_col,
+                desc_remaining - 1,
+                false,
+                &mut line_data,
+            );
+            desc_remaining -= print_max(
+                offset_in_cmdline,
+                L!(")"),
+                paren_col,
+                1,
+                false,
+                &mut line_data,
+            );
             let _ = desc_remaining;
         } else {
             // No description, or it won't fit. Just add spaces.
             print_max(
+                offset_in_cmdline,
                 &WString::from_iter(std::iter::repeat(' ').take(desc_remaining)),
                 bg,
                 desc_remaining,
@@ -1062,6 +1101,7 @@ fn divide_round_up(numer: usize, denom: usize) -> usize {
 /// \param has_more if this flag is true, this is not the entire string, and the string should be
 /// ellipsized even if the string fits but takes up the whole space.
 fn print_max_impl(
+    offset_in_cmdline: usize,
     s: &wstr,
     color: impl Fn(usize) -> HighlightSpec,
     max: usize,
@@ -1082,13 +1122,13 @@ fn print_max_impl(
 
         let ellipsis = get_ellipsis_char();
         if (width_c == remaining) && (has_more || i + 1 < s.len()) {
-            line.append(ellipsis, color(i));
+            line.append(ellipsis, color(i), offset_in_cmdline);
             let ellipsis_width = wcwidth_rendered(ellipsis);
             remaining = remaining.saturating_sub(usize::try_from(ellipsis_width).unwrap());
             break;
         }
 
-        line.append(c, color(i));
+        line.append(c, color(i), offset_in_cmdline);
         remaining = remaining.checked_sub(width_c).unwrap();
     }
 
@@ -1096,8 +1136,15 @@ fn print_max_impl(
     max.checked_sub(remaining).unwrap()
 }
 
-fn print_max(s: &wstr, color: HighlightSpec, max: usize, has_more: bool, line: &mut Line) -> usize {
+fn print_max(
-    print_max_impl(s, |_| color, max, has_more, line)
+    offset_in_cmdline: usize,
+    s: &wstr,
+    color: HighlightSpec,
+    max: usize,
+    has_more: bool,
+    line: &mut Line,
+) -> usize {
+    print_max_impl(offset_in_cmdline, s, |_| color, max, has_more, line)
 }
 
 /// Trim leading and trailing whitespace, and compress other whitespace runs into a single space.

src/reader.rs

@@ -83,6 +83,7 @@ use crate::input_common::{
     ReadlineCmd,
 };
 use crate::io::IoChain;
+use crate::key::ViewportPosition;
 use crate::kill::{kill_add, kill_replace, kill_yank, kill_yank_rotate};
 use crate::libc::MB_CUR_MAX;
 use crate::nix::isatty;
@@ -491,7 +492,9 @@ pub struct ReaderData {
     last_flash: Option<Instant>,
 
     /// The representation of the current screen contents.
-    screen: Screen,
+    pub screen: Screen,
+
+    pub pending_mouse_left: Option<ViewportPosition>,
 
     /// Data associated with input events.
     /// This is made public so that InputEventQueuer can be implemented on us.
@@ -1144,6 +1147,7 @@ impl ReaderData {
             first_prompt: true,
             last_flash: Default::default(),
             screen: Screen::new(),
+            pending_mouse_left: None,
             input_data,
             queued_repaint: false,
             history,
@@ -1347,6 +1351,22 @@ impl ReaderData {
         }
         true
     }
+
+    pub fn mouse_left(&mut self, cursor: ViewportPosition) {
+        let click_position = self.pending_mouse_left.take().unwrap();
+        FLOG!(
+            reader,
+            "Cursor is at",
+            cursor,
+            "; received left mouse click at",
+            click_position
+        );
+        let new_pos = self
+            .screen
+            .offset_in_cmdline_given_cursor(click_position, cursor);
+        let (elt, _el) = self.active_edit_line();
+        self.update_buff_pos(elt, Some(new_pos));
+    }
 }
 
 /// Given a command line and an autosuggestion, return the string that gets shown to the user.

src/screen.rs

@@ -7,6 +7,7 @@
 //! The current implementation is less smart than ncurses allows and can not for example move blocks
 //! of text around to handle text insertion.
 
+use crate::key::ViewportPosition;
 use crate::pager::{PageRendering, Pager, PAGER_MIN_HEIGHT};
 use std::cell::RefCell;
 use std::collections::LinkedList;
@@ -42,6 +43,8 @@ use crate::wutil::fstat;
 pub struct HighlightedChar {
     highlight: HighlightSpec,
     character: char,
+    // Logical offset within the command line.
+    offset_in_cmdline: usize,
 }
 
 /// A class representing a single line of a screen.
@@ -64,17 +67,18 @@ impl Line {
     }
 
     /// Append a single character `txt` to the line with color `c`.
-    pub fn append(&mut self, character: char, highlight: HighlightSpec) {
+    pub fn append(&mut self, character: char, highlight: HighlightSpec, offset_in_cmdline: usize) {
         self.text.push(HighlightedChar {
             highlight,
             character: rendered_character(character),
+            offset_in_cmdline,
         })
     }
 
     /// Append a nul-terminated string `txt` to the line, giving each character `color`.
-    pub fn append_str(&mut self, txt: &wstr, highlight: HighlightSpec) {
+    pub fn append_str(&mut self, txt: &wstr, highlight: HighlightSpec, offset_in_cmdline: usize) {
         for c in txt.chars() {
-            self.append(c, highlight);
+            self.append(c, highlight, offset_in_cmdline);
         }
     }
 
@@ -93,6 +97,11 @@ impl Line {
         self.text[idx].highlight
     }
 
+    /// Return the logical offset corresponding to this cell
+    pub fn offset_in_cmdline_at(&self, idx: usize) -> usize {
+        self.text[idx].offset_in_cmdline
+    }
+
     /// Append the contents of `line` to this line.
     pub fn append_line(&mut self, line: &Line) {
         self.text.extend_from_slice(&line.text);
@@ -321,6 +330,7 @@ impl Screen {
         // Append spaces for the left prompt.
         for _ in 0..layout.left_prompt_space {
             let _ = self.desired_append_char(
+                /*offset_in_cmdline=*/ 0,
                 usize::MAX,
                 ' ',
                 HighlightSpec::new(),
@@ -364,6 +374,7 @@ impl Screen {
                 break scrolled_cursor.unwrap();
             }
             if !self.desired_append_char(
+                /*offset_in_cmdline=*/ i,
                 if is_final_rendering {
                     usize::MAX
                 } else {
@@ -477,6 +488,29 @@ impl Screen {
         self.r#move(0, self.actual.line_count());
     }
 
+    pub fn offset_in_cmdline_given_cursor(
+        &mut self,
+        viewport_position: ViewportPosition,
+        viewport_cursor: ViewportPosition,
+    ) -> usize {
+        let viewport_prompt_y = viewport_cursor.y - self.actual.cursor.y;
+        let y = viewport_position.y - viewport_prompt_y;
+        let y = y.min(self.actual.line_count() - 1);
+        let viewport_prompt_x = viewport_cursor.x - self.actual.cursor.x;
+        let x = viewport_position.x - viewport_prompt_x;
+        let line = self.actual.line(y);
+        let x = x.max(line.indentation);
+        if x >= line.len() {
+            if self.actual.line_count() == 1 {
+                0
+            } else {
+                line.text.last().unwrap().offset_in_cmdline + 1
+            }
+        } else {
+            line.offset_in_cmdline_at(x)
+        }
+    }
+
     /// Resets the screen buffer's internal knowledge about the contents of the screen,
     /// abandoning the current line and going to the next line.
     /// If clear_to_eos is set,
@@ -598,6 +632,7 @@ impl Screen {
     /// automatically handles linebreaks and lines longer than the screen width.
     fn desired_append_char(
         &mut self,
+        offset_in_cmdline: usize,
         max_y: usize,
         b: char,
         c: HighlightSpec,
@@ -623,6 +658,7 @@ impl Screen {
             line.indentation = indentation;
             for _ in 0..indentation {
                 if !self.desired_append_char(
+                    offset_in_cmdline,
                     max_y,
                     ' ',
                     HighlightSpec::default(),
@@ -660,7 +696,9 @@ impl Screen {
                 self.desired.cursor.x = 0;
             }
 
-            self.desired.line_mut(line_no).append(b, c);
+            self.desired
+                .line_mut(line_no)
+                .append(b, c, offset_in_cmdline);
             self.desired.cursor.x += cw;
 
             // Maybe wrap the cursor to the next line, even if the line itself did not wrap. This
@@ -936,7 +974,7 @@ impl Screen {
             zelf.r#move(0, 0);
             let mut start = 0;
             let osc_133_prompt_start =
-                |zelf: &mut Screen| zelf.write_bytes(b"\x1b]133;A;special_key=1\x07");
+                |zelf: &mut Screen| zelf.write_bytes(b"\x1b]133;A;click_events=1\x07");
             if left_prompt_layout.line_breaks.is_empty() {
                 osc_133_prompt_start(&mut zelf);
             }