4 commits

Author	SHA1	Message	Date
Karl H	96d7de6db1	mbsync: per account multiple channels (#1360) ... * mbsync: option for configuring a channel A channel is a relationship between 2 directories/boxes/mailboxes between the local machine (slave) and the remote mail server (master). Each channel must be given at least: * an account-unique name * a pattern for which mailboxes to sync from master * a pattern for what directory where that mail ends up on the slave Additional options can be added later. * mbsync: option for configuring a group A group is a grouping of channels together, so that many channels with very different names can be handled as a single entity. Groups are unique in mbsync because they will shadow channels that have the same name on the command-line. * mbsync: create groups configuration attribute This is the end of the configuration that the end-user will use. They will specify an attribute set that contains the name for the group, so they can say `accounts.email.accounts.<aname>.groups.<gname>` to access the configuration for the group with the name `<gname>`. * mbsync: write function to generate group-channel blocks This function takes in a set of groups, and their consituent channels and writes the appropriate .mbsyncrc block. The block is as shown below: Group groupName1 Channel channelName1 Channel channelName2 Group groupName2 Channel channelName3 Each group must have a unique name, no matter which account it is declared under. The same holds true for channels. However, if there is a group that shares the same name as the channel, the channel will effectively be "shadowed" by the group, and mbsync will default to working with the group in that case. * mbsync: write function to generate channel configuration blocks This function takes in a set of groups, which includes their consituent channels and writes the appropriate .mbsyncrc block for the channel. The block that is generated is shown below: Channel groupName1-channelName1 Master :<accountName>-remote:<master-pattern> Slave :<accountName>-local:<slave-pattern> Channel groupName2-channelName2 Master :<accountName>-remote:<master-pattern> Slave :<accountName>-local:<slave-pattern> Each group must have a unique name, no matter which account it is declared under. The same holds true for channels. Using channels with the patterns set up this way allows one to specify which maildir directories are to be synchronized FROM the master TO the slave. In addition, it allows for these maildirs to be remapped, between the master server and the local slave. This is critical, because Gmail has a strange way of storing its mail that makes using mbsync, mu, and mu4e more difficult. There are additional channel parameters that are already present in this codebase from the previous use of group-channel configuration, which will be reused. * mbsync: set the submodule's names field according to parameter This is the same method as is used in creating an email account, named `<name>` under `accounts.email.accounts.<name>`. This allows the user to specify groups and channels, in a list-like format, but still gets the "namespacing" to more easily handle the options available in each of these locations. * mbsync: provide examples of master/slave patterns for channels * mbsync: create nested-let function to generate channel pattern This pattern is required to either NOT be present, which means the master pattern is used to match, or it has a list of patterns to use beneath the master maildir to match against. This function checks to ensure that if patterns is not empty, ONLY then is the `Pattern` keyword printed. Otherwise, there are many, many problems. If there IS a list of patterns, then we use proper escaping methods to ensure that the exact string is constructed. * mbsync: per-account groups can have additional patterns Gave the `accounts.email.accounts.<name>.mbsync.groups.<gname>.channel.<cname>` set a `patterns` option, which will allow for greater customization and filtering of the master maildir to sync to the slave maildir. * mbsync: add extraConfig option for easier-to-format options These are options that can be handled by the `genSection` function in the `genAccountFunction`, so they are left to the user to decide. Most of these are made on a global basis anyways. * mbsync: remove unneeded extraConfig.channel This was originally placed here, seemingly, just to get this module working. However, this field is actually more confusing now that a separate per-channel configuration option for extra configurations has been made available. * mbsync: correct and improve comment in masterPattern description * mbsync: switch channel/group generation to new functions Changing this out is what moves us from the old system to the new one. Instead of having a single channel manage a whole mailbox, we can now specify an attribute set of groups that should correspond to an email account. Each of these groups contains an attribute set of channels that make it up, and are grouped together for synchronization. In addition, each of these channels can have additional IMAP4 parameters attached to them to further refine synchronization. Lastly, each of the channels is grouped together under the Group section, ensuring that the channels' mailboxes synchronize as they have been specified. * mbsync: only generate group/channel configuration if channels present Typically, when a group is specified, channels will be specified as well. However, if due to error or mistake, the user forgets to specify ANY channels for a group, we should not generate that group's information. This means that no channels are specified (which maps the remote master to local slave). In addition, the `Group <gName>` block (which brings the separate channels together) is also not generated. Another thing to consider is that a user might specify a group and a channel, but perform no additional configuration of the channel. In a configuration, this would be realized by `accounts.email.accounts.<aName>.mbsync.groups.<gName>.channels.<cName>;` This creates the channel with the name `<cName>` and the `masterPattern`, `slavePattern`, and `patterns` fields use their defaults. By definitions set within mbsync, these defaults actually specify that the remote master's `INBOX` mail directory is synchronized to the local slave's `INBOX` directory. So, if there is a channel that has no fields specified, then we DO want to generate its configuration. But if there is a group that has no channels, then we do NOT generate it. * mbsync: acc comment explaining why groups attr set is never empty * Revert "mbsync: remove unneeded extraConfig.channel" This reverts commit 941c4771ca. To support backwards compatibility, I need to leave this field/option in the module, even if it will likely be more confusing to do it this way. * mbsync: channel compatibility with previous iteration of mbsync The previous version of mbsync used a single channel for an entire account. This leads to issues when trying to change the mailbox hierarchy on the local machine. The problem with this is that some email providers (Gmail, among others) use a slightly different maildir hierarchy, where the standard mailboxes (Inbox, Drafts, Trash, etc.) are stored inside another directory (`[Gmail]/` in the case of Gmail). This new version allows the user to specify any number of groups with any number of channels within to reorder their mail however they wish. However, to maintain backwards compatibility, I moved the original channel-generating code to a function that will run ONLY when there are no groups specified for THIS account. * Revert "mbsync: channel compatibility with previous iteration of mbsync" This reverts commit b1a241ff9f. This function is in the wrong location and this was wrongly committed. * mbsync: function for backwards compatibility with previous mbsync NOTE THAT THIS IS THE CORRECT COMMIT FOR THIS CHUNK OF CODE!! The previous version of mbsync used a single channel for an entire account. This leads to issues when trying to change the mailbox hierarchy on the local machine. The problem with this is that some email providers (Gmail, among others) use a slightly different maildir hierarchy, where the standard mailboxes (Inbox, Drafts, Trash, etc.) are stored inside another directory (`[Gmail]/` in the case of Gmail). This new version allows the user to specify any number of groups with any number of channels within to reorder their mail however they wish. However, to maintain backwards compatibility, I moved the original channel-generating code to a function that will run ONLY when there are no groups specified for THIS account. * mbsync: function to choose which style of group/channels to generate This is a simple if-check. If the old style is used, then this account's mbsync.groups attribute set is empty. If that is the case, then the old-style single-channel per account is used. If that is NOT the case, then the new style is used in preference of the old. This means that ALL channel code that would be generated by the old version is replaced by the new one. * mbsync: switch per-account config generation to check channels * mbsync: program-wide groups if no account-specific groups At the end, we have to choose whether or not to generate the old style of having program-wide groups to specify things, where the boxes on the channel underneath the group specifies which mailboxes to sync. Here, we only generate the old style of group IF there is ANY account that does NOT have the new `accounts.mbsync.groups` defined. At that point, it is up to the user to ensure that the accounts in `programs.mbsync.groups.{}` align with the name chosen for the account, as I have made no attempt to change this old code. However, if ALL accounts have their `mbsync.groups` defined, even if each of the groups has a single empty channel, it will generate the groups in the new style. * mbsync: ensure \n after hm-generated comment This was a multi-part fix. First, the `# Generated by Home Manager.` comment has been reworked to ensure that it will ALWAYS have a newline, even if the program-wide extraConfiguration is empty. Next, we switched to placing 2 newlines between every account, to provide further visual distinction between each account, which can have multiple channels and multiple groups defined at the same time. Lastly, the groupsConfig was slightly reworked, so that both the old and new version can be used, but the new one will take precedence. Because of this, groupsConfig is now a list of strings, which will have single newlines inserted between each element. But if the old style is NOT used, then the groupsConfig list contains one element, an empty string. A single element has nothing added as a separator, and an empty string produces no output. * mbsync: only generate new group/channels if channels present Here, the problem was if the user created a group for an account, but did not also include a set of channels. If no channels have been specified, then the group should NOT have its group-channel mapping generated. I also corrected and improved the comment regarding `genGroupChannelString`'s function and intended behavior. * mbsync: channel patterns generate their own newlines This means that when a channel has extra `patterns` defined for it, it will generate those, and a single newline will be appended to the end of that newly constructed string. The moving of the newline character is slightly important because otherwise, every account would receive an extra newline after every channel, leading to 2 newlines after every channel. * mbsync: place newline between each channel in a group * mbsync: ensure old group/channel has proper spacing This ensures that if the old style of generating program-wide groups that there is the proper spacing before the group and in between each line within the group. * mbsync: ensure no empty channels present If the user specifies a group correctly, they must still specify an attribute set of channels. However, if they do not, then we need to ensure that a group with no channels does NOT have any channel configurations generated for it. If there is a channel string generated for a channel that is empty, then the `mapAttrsToList` returns a singleton list that contains just the empty string. Thus, we can filter out all those results, to ensure that no empty channels are generated. It is important to keep in mind the difference between an empty channel and a channel that has received no configuration, but is named. * A named channel is technically configured to have a name. While the `masterPattern`, `slavePattern`, and `patterns` field have NOT been populated, mbsync assumes that if master/slave-Pattern are empty that means match against `INBOX`. If `patterns` is empty, no patterns are printed. * An empty channel set is a set that has no channels within it, but `mbsync.groups.<gName>.channels` is defined. * mbsync: filter empty groups and correct newlines First thing, someone can specify that a group is empty. If this is done, technically a group with channels would be generated at the end. However, because they were empty and did not exist, whitespacing would be generated, leading to a usable, but mangled config file. The `filter` solves this problem by removing empty strings (which are generated by groups that are empty) from the output strings to place in the file. Lastly, because the whitespacing was fixed elsewhere in the file, the crazy double-newline at the end was changed to a single newline. However, the double newline within the `concatStringsSep` is still required, because the list that is being concatenated together is a list of channel configurations. Each element corresponds to one of the groups specified, whose contents are the channels specified within. The double newline is needed because each string element is lacking a trailing newline, because `concatStringsSep` does not add the separator to the end of the last element in the list. So, the last channel to be configured will not have that newline appended when the channel-configuration list is created, thus, 2 are inserted here. * mbsync: update test input to use per-account channels * mbsync: comment how old/new style collision handled This is left in the test input for now, because I think it is useful to see why certain things are happening the way they are. * mbsync: update test output pattern The test output should now have the correct configuration according to the way I have specified it in the input file. * mbsync: use format script on new code * mbsync: add KarlJoad as maintainer Co-authored-by: Nick Hu <me@nickhu.co.uk>	2020-09-21 18:16:06 +01:00
Robert Helgesson	95b95b1407	tests: improve test purity ... Before the XDG variables would be set from the user's environment, if available. This would break some tests. With this change the tests should be fully deterministic. Fixes #1190	2020-04-24 21:46:34 +02:00
Robert Helgesson	45abf3d38a	Apply nixfmt on many files	2020-02-02 01:07:28 +01:00
Robert Helgesson	de8033747c	tests: clean up tests ... - Move all module tests to their own directories. - Avoid duplication of `// import`.	2020-01-26 21:11:23 +01:00