This improves performance by a factor of at least 2 in large --tree workloads by avoiding the repeated creation/destruction of the pool and containing threads.
Cycling pools also encountered lots lock contention, which accounted for most of the time saved by reusing a single pool.
Add two new sort options `.name` and `.Name` which with ignore a leading
`.` if present on the file name before sorting according to `name` and
`Name`.
This new sort is convenient if you want to list hidden and unhidden
files sorted together.
Fixes#288, but more-or-less as a side-effect.
The “mi” key in LS_COLORS was meant to be used for a missing link path, but it wasn’t really used like that. There was also a bug where control characters in a broken symlink’s path were assumed to be underlined, because that’s what happened in the default colour scheme, but this assumption doesn’t hold when colours were disabled.
The solution to these was not to introduce another configurable colour code, but to start using _overlays_ to alter a bunch of colours at once. The “mi” code will have to be added back later.
This doesn’t *completely* work: it seems to have trouble with ignored paths beginning with slashes, possibly amongst others. Also, .gitignore scanning could be made more efficient.
I changed my mind about which way round sorting by “newest” or by “oldest” should actually go. If you’re listing a large directory, you see the last lines of the output first, so these files should be the ones with the largest whatever the sort field is. It’s about sorting *last*, not sorting *first*. Sorting by size wouldn’t say “sorts smallest files first”, it would say “sorts largest files last”. Right?
Also, add a new suggestion that warns against “ls -lt”.
This commit changes the definition of Arg so that it knows about which values it can accept, and can display them in the help text. They were already being shown in the help text, but they were passed in separately, so one argument could show two different sets of options if it wanted. Now, the argument itself knows whether there are suggestions, so it doesn’t have to be passed in separately.
This means we can use it for other things, including listing choices when an option is missed out, without having to repeat the list.
With Misfire::BadArgument now only having two fields, it’s not worth using a constructor function anymore.
TakesValue::Optional introduced which allows for an optional flag with
an optional value (equivalent to getopts' optflagopt mode).
Can be used where a default value for a modifier could exist, but the
user might prefer to override.
Will be used to implement #284, permitting --time to default to "sort by
modification date" for compatibility with GNU/posix ls but keeping
support for exa's previous behavior.
Raised in #243 and #284. exa isn’t able to override the -t option like this, so the least it can do is detect that case (which is going to be an error case anyway) and show a suggestion.
I don’t really see the modified date as the *modified* date, rather just the *date* field, because it’s the date field I refer to like 99.9% of the time. So now it has aliases to match.
Also are included are aliases for the reverse order, because I’d rather write “new” than “the reverse of old”.
It said “(Choices: (choices: this, that, other))” instead of “(choices: this, that, other)”. Also improve the same error elsewhere: options more have ‘settings’ than ‘values’.
This commit adds to the parsing of the LS_COLORS and EXA_COLORS variables so that non-two-letter codes (keys other than things like ‘di’ or ‘ln’ or ‘ex’) will be treated as file name globs, and get used to colour files accordingly.
Fixes#116 for good.
This is more annoying than it should be because it has to work with Styles rather than with strings, which means parsing them, and parsing is always tricky business.
This is all a big commit because it took a lot more work than I thought it would! The commit basically moves Git repositories from being per-directory to living for the whole life of the program. This allows for several directories in the same repository to be listed in the same invocation; before, it would try to rediscover the repository each time! This is why two of the tests “broke”: it suddenly started working with --recurse.
The Dir type does now not use Git at all; because a Dir doesn’t have a Git, then a File doesn’t have one either, so the Git cache gets passed to the render functions which will put them in the Table to render them.
This uses the Git module’s newfound powers of getting actual GitRepo values from a factory to cache repositories a bit more. Now, when querying two directories under the same repository, it’ll open both, see that they have the same workdir, and only use the first one.
This commit adds a cache for Git repositories based on the path being queried.
Its only immediate effect is that when you query the same directory twice (such as /testcases/git /testcases/git), it won’t need to check that the second one is a Git directory the second time. So, a minuscule optimisation for something you’d never do anyway? Wrong! It’s going to let us combine multiple entries over the same repository later, letting us use --tree and --recurse, because now Git scanning is behind a factory.
This adds support for the EXA_COLORS environment variable, and defines a bunch of exa-specific two-letter codes that I pretty much made up arbitrarily that control parts of the interface.
Fixes#160, which I didn’t expect to actually fix this release cycle, but it unexpectedly became easy to do!
LSColors used to be built up from an iterator, and then queried later. But because the resulting HashMap gets queried in serial anyway, we might as well pass in a callback instead, saving the allocation.
This is also technically a little faster because styles that don’t map to anything (like `zz`) are no longer parsed.
This commit meddles about with both the Colours and the FileExtensions.
Even though all the renderable fields were turned into traits, the FileName struct kept on accessing fields directly on the Colours value instead of calling methods on it. It also did the usual amount of colour misappropriation (such as ‘punctuation’ instead of specifying ‘normal_arrow’)
In preparation for when custom file colours are configurable (any day now), the colourise-file-by-kind functionality (links, sockets, or directories) was separated from the colourise-file-by-name functionality (images, videos, archives). The FileStyle struct already allowed for both to be separate; it was only changed so that a type other than FileExtensions could be used instead, as long as it implements the FileColours trait. (I feel like I should re-visit the naming of all these at some point in the future)
The decision to separate the two means that FileExtensions is the one assigning the colours, rather than going through the fields on a Colours value, which have all been removed. This is why a bunch of arbitrary Styles now exist in filetype.rs.
Because the decision on which colourise-file-by-name code to use (currently just the standard extensions, or nothing if we aren’t colourising) is now determined by the Colours type (instead of being derived), it’s possible to get it wrong. And wrong it was! There was a bug where file names were colourised even though the rest of the --long output wasn’t, and this wasn’t caught by the xtests. It is now.
There are now two device colours instead of one. Even though they’re both set to the same style for the default colour set, LS_COLORS allows the two to look different, so exa has to support it too.
It’s probably a good idea to support it anyway.
This separates the colours to give to files with different filesystem types (directories, links, sockets) from files with different names or extensions (images, videos, archives).
I’m not 100% sure I’ve got the terms “kind” and “type” the right way round, but whatever.
This was done because colouring files based on their name is going to be handled differently and extensibly from colouring files based on what the filesystem thinks.
This commit adds many traits, all named ‘Colours’, to the code. Each one asks for a colour needed to render a cell: the number of links asks for colours for the number and the multi-link-file special case; the file size asks for number, unit, punctuation, and device ID colours, or it can do a scale with its own colours, however it wants.
This is a step towards LS_COLORS compatibility, believe it or not. If a text cell in a column doesn’t depend on Colours to render itself, then the source of the colours is open-ended.
I am glad to have not needed any test changes here.
This was touched on in #209 where I got the docs wrong compared to the actual implementation, but after thinking about it, I’d like to switch it round. (The --sort=Name and --sort=name difference has also been switched.) See the big ol’ comment for my reasons.
Because this changes core functionality, it broke many, many tests. You can see that this doesn’t change the -star- tests because the shell, rather than exa, orders the globbed files.
I kept on forgetting which way round Sensitive and Insensitive went, so I named them after the effect they have.
The Debug impl was being used instead of the Display one. Also, remove the full stops from the ends of all the error messages because I’ve decided it looks weird.
The error in #178 was being hidden from output most of the time, and because exa isn’t a GUI program, there’s nowhere it can really dump log output like this. Now that users can opt in with the EXA_DEBUG variable, there is a place it can go.
Also stop the ERROR log level being printed by default.
I want to be very careful when doing the “--git and --tree don’t work together” one to not search for more Git repositories than I should. Being able to log when a repository is looked up and also switch that functionality on and off would help with that.
Just because the type that gets used right now is Copy and Clone doesn’t mean that when we pass mock ones in for tests they’ll be those two as well. So we have to go through and add &s everywhere.
No new features here, just some restructuring. Mode::GridDetails was nice and elegant with those two fields, but now there’s a grid-details-only option the elegance has gone out the window.
Yeah, I forgot what I was meant to be doing half-way through.
This also adds the row_threshold field, which disables the view unless there will be more than the given number of rows. Getting the row count required upgrading term_grid to a version that has that function added.
Previously the iterator went all the way through `2..`, and not only would that take a very long time, but at the end it wouldn’t even print anything. Now the grid-details view turns into a lines view when it’s hit its limit.
exa now ignores errors when checking for extended attributes when the user didn’t explicitly demand that they be checked. If a file does have xattrs, it’ll still display the @ in the permissions column; errors will now just cause the @ to be hidden instead.
This changed a lot of the xtests, which were displaying the error message in a few situations. Those tests have gained @-suffixed companions so the actual error messages can still be tested.
Fixes#178 (finally)
It’s a good test to be able to switch strict mode on in run.sh and not have it break anything! Now, the EXA_STRICT environment variable will toggle it on. We can even switch it off and see that it doesn’t error.
Some of the deduce functions used to just blatantly call std::env::var_os and not care, introducing global state into a module that was otherwise nice and functional and self-contained. (Well, almost. There’s still terminal width.)
Anyway, this made it hard to test, because we couldn’t test it fully with this global dependency in place. It *is* possible to work around this by actually setting the environment variables in the tests, but this way is more self-documenting.
With this in place, we can start to unit test things like deriving the view by passing in what the $COLUMNS environment variable should be, and that’s one of the first things checked.
src/options/mod.rs *almost* has all its tests moved to where they should be!
Sometimes, the type in the Ok part of the Result wouldn’t implement PartialEq, so the first macro (which uses assert_eq) won’t work. In these cases, this new macro can be used instead, which just unwraps the Err’s contents. In other cases, it can shave off a ) at the end of a few lines.
The table Options struct is roughly half runtime configuration and half flags to select which columns to display The column fields might as well be in their own struct, and now that the ‘for_dir’ function doesn’t use SizeFormat, it can be moved to Columns.
Way in the past, the size format was the only variable column; the others were all fixed. Now there are many configurable columns and this field was still hanging around. The code that does the rendering just gets the size format as an argument, and now it works the same way as the TimeFormat.
This changes the SizeFormat option parser from its old, strict-by-default behaviour (where passing both --bytes and --binary would be an error) to the new, use-the-last-argument behaviour (where passing --bytes --binary would use --binary because it came later).
Doing this meant adding functionality to Matches so that it could return *which* argument matched. Previously, the order of --bytes and --binary didn’t matter, because they couldn’t both be present, but now it does.
The assert_parses function was problematic because it insisted on using assert_eq! to check its contents. This won’t work for any type we want to test that doesn’t implement PartialEq, such as TimeFormat, which holds references to years and date strings and other such.
To go about fixing this, the first step is to change that function so it only does the initial processing, rather than the assertion, which is now done outside of it in the test macros instead.
Now the code actually starts to use the Strictness flag that was added in the earlier commit! Well, the *code* doesn’t, but the tests do: the macros that create the test cases now have a parameter for which tests they should run. It’s usually ‘Both’ for both strict mode and default mode, but can be specified to only run in one, for when the results differ (usually when options override one another)
The downside to strict mode is that, now, *any* call to `matches.has` or `matches.get` could fail, because an option could have been specified twice, and this is the place where those are checked for. This makes the code a little less ergonomic in places, but that’s what the ? operator is for. The only place this has really had an effect is in `Classify::deduce`, which used to just return a boolean but can now fail.
In order to more thoroughly test the mode, some of the older parts of the code can now act more strict. For example, `TerminalColours::deduce` will now use the last-given option rather than searching for “colours” before “colors”.
Help and Version continue doing their own thing.
The value is ignored, but this broke quite a lot of tests that assumed MatchedFlags had only one field.
Parsing tests have to have OsStr flags because I couldn’t get that part working right, but in general, some tests now re-use common functionality too.
This commit gives IgnorePatterns a bunch of constructor methods that mean its option-parsing sister file doesn’t need to know that it’s a vec of glob patterns inside: it can work with anything that iterates over strings. Now, the options module doesn’t need to know about the glob crate.
This merges in exa’s own new options parser, which has the following features:
- You can specify an option twice and it’ll use the second one, making aliases usable for defaults (fixes#144)
- Lets arguments be specified more than once (fixes#125)
Strict mode is not done yet; I just wanted to merge this in because it’s been a while, and there’s work that needs to be done on master so I don’t want them drifting apart any further.
It’s likely that you’ll find cases where multiple arguments doesn’t work or where the wrong value is being used. There aren’t tests for *everything* yet, and it still uses global environment variables.
# Conflicts:
# src/options/view.rs
The term_size crate introduced in #237 did things *slightly* differently than exa: it tried to get the terminal width of stdout, stderr, and stdin. This broke some tests that only redirected stdout.
Now it’s more like help. There aren’t any other fields in its struct at the moment, but there will be in the future (listing the features, and extremely colourful vanity mode)
Originally, both the matched flags and the list of free strings were returned from the parsing function and then passed around to every type that had a ‘deduce’ method. This worked, but the list of free strings was carried around with it, never used.
Now, only the flags are passed around. They’re in a new struct which has the methods the Matches had.
Both of Matches’s fields are now just data, and all of the methods on MatchedFlags don’t ignore any fields, so it’s more cohesive, at least I think that’s the word.
Building up the MatchedFlags is a bit more annoying though because the vector is now hidden behind a field.
One of the previous tests started to fail, because it was working when it shouldn’t have! It worked up until now because I forgot to flag --level as taking an argument, and “--level 4” still worked with 4 as a filename. So there’s now an early check for that functionality that got lost somewhere.
Apparently I forgot to give the --time flag an argument, and this wasn’t actually covered by any of the xtests! Well, it’s tested now.
I’m not sure how to handle multiple --time arguments.
This commit removes the dependency on the ‘getopts’ crate entirely, and re-writes all its uses to use the new options parser instead.
As expected there are casualties galore:
- We now need to collect the options into a vector at the start, so we can use references to them, knowing they’ll be stored *somewhere*.
- Because OsString isn’t Display, its Debug impl gets used instead. (This is hopefully temporary)
- Options that take values (such as ‘sort’ or ‘time-style’) now parse those values with ‘to_string_lossy’. The ‘lossy’ part means “I’m at a loss for what to do here”
- Error messages got a lot worse, but “--tree --all --all” is now a special case of error rather than just another Misfire::Useless.
- Some tests had to be re-written to deal with the fact that the parser works with references.
- ParseError loses its lifetime and owns its contents, to avoid having to attach <'a> to Misfire.
- The parser now takes an iterator instead of a slice.
- OsStrings can’t be ‘match’ patterns, so the code devolves to using long Eq chains instead.
- Make a change to the xtest that assumed an input argument with invalid UTF-8 in was always an error to stderr, when that now in fact works!
- Fix a bug in Vagrant where ‘exa’ and ‘rexa’ didn’t properly escape filenames with spaces in.
This commit moves the definitions of Filter and DirAction from the options module to the fs module, but leaves the parts that actually have to do with option parsing alone.
Now, the options module shouldn’t define any types that get used elsewhere in the program: it only adds functionality to types that already exist.
Casualty here was that you can’t have static values reference one another directly, so the static args slice had to be turned into a slice *of references* rather than of values. No big deal, just have to write & a few more times.
The FileExtensions in the FileName is now a reference to the one in the original FileStyle, which gets put there in the options module.
This allows the extensions to be derived from the user, somehow, in the future when that part’s done.
Instead of having a File do its own extension checking, create a new type that takes a file and checks *that*. This new type (FileExtensions) is currently empty, but were it to contain values, those values could be used to determine the file’s colour.
This commit replaces the “two normal cases” of showing a link’s target or not with “one default and one special case” of preferring to hide them, displaying the link targets by setting a flag instead.
Doing this simplifies the file name constructor, which gets to remove an argument.
The new FileStyles value will contain all the fields necessary to “style” a file’s name. Right now this is only the Classify field, but there can be more later. The benefit of this is that when we add more, we won’t need to update all the places where file names are displayed.
This commit moves the Environment field from the Table to its Options, and properly gets rid of the name ‘columns’ from the last commit.
Having it in the Options is important, because it means it can be generated from some command-line options. Also, it reduces the number of arguments that need to be passed to Table::new; there would have been 4 with the inclusion of the Environment, but by moving some of the code into the function, we can avoid this (and any further arguments).
The views have been renamed to be the Optionses of their module; now the options for the Table — Columns — has followed suit.
This works out, because the table module depended on everything in the columns module. It opens the door for other only-table-specific things to be included.
The casualty was that by making it non-Clone and non-PartialEq, a bunch of other #[derive]-d types had to have their derivions removed too.
The Environment struct only used the Default trait so it could have the same call for both Environment<UsersCache> and Environment<MockUsers>. There’s no reason to keep it around anymore.
There was a bug where if you tried to recurse into a directory you didn’t have permission to read the contents of, the error would be ignored.
It now displays the errors.
The goal of this part of the refactoring, if you wondered, is to make it so only the tree module is aware that it needs ‘depth’ and ‘last’ values to draw the tree.
As far as the details module is concerned, it should just be doing something to produce TreeParams values which it later consumes; that’s it.
This change should make it easier to have tables that may or may not have a tree in them.
Adding a header row automatically added the widths to the table and returned the row, but adding a file’s row didn’t add the widths. Now they’re consistent.
By having the widths be in a separate type, we can separate the two out later, rather than having one refer to the other.
This commit ties a table’s Environment to the fact that it contains columns.
Previously, the Details view would get its Environment, and then use those fields to actually display the details in the table: except for the case where we’re only displaying a tree, when it would just be ignored, instead.
This was caused by the “no columns” case using a Vec of no Columns behind the scenes, rather than disabling the table entirely; much like how a tap isn’t a zero-length swipe, the code should have been updated to reflect this. Now, the Environment is only created if it’s going to be used.
Also, fix a double-mutex-lock: the mutable Table had to be accessed under a lock, but the table contained a UsersCache, which *also* had to be accessed under a lock. This was changed so that the table is only updated *after* the threads have all been joined, so there’s no need for any lock at all. May fix#141, but not sure.
This commit extracts the common table element from the details and grid_details modules, and makes it its own reusable thing.
- A Table no longer holds the values it’s rendering; it just holds a continually-updated version of the maximum widths for each column. This means that all of the resulting values that turn into Rows — which here are either files, or file eggs — need to be stored *somewhere*, and that somewhere is a secondary vector that gets passed around and modified alongside the Table.
- Likewise, all the mutable methods that were on Table that added a Row now *return* the row that would have been added, hoping that the row does get stored somewhere. (It does, don’t worry.)
- Because rendering with mock users is tested in the user-field-rendering module, we don’t need to bother threading different types of U through the Environment, so now it’s just been specialised to UsersCache.
- Accidentally speed up printing a table by not buffering its entire output first when not necessary.
This isn’t perfect, as a file’s type isn’t cached, so it gets recomputed for every comparison in the sort! We can’t go off the file’s `st_mode` flag because it’s not guaranteed to be in any order between systems.
The arguments passed to File’s constructor were different from the field names used — these might as well both be the same.
Also, move ext and filename to be File methods to save an import, and add tests.
Also also, by passing a PathBuf in to the constructor directly, we can save one (possibly two) instance/s where we pass in a reference to something we were going to lose ownership of anyway, only to have it basically cloned.
There’s a problem with the tree view where it’ll still recurse through `.` and `..`. But if you were using tree view, would you even need to see them? They’d be in the tree already!
There was a problem when displaying . and .. in directory listings: their names would normalise to actual names! So instead of literally seeing `.`, you’d see the current directory’s name, inserted in sort order into the list of results. Obviously this is not what we want.
In unrelated news, putting `.` and `..` into the list of paths read from a directory just takes up more heap space for something that’s basically constant.
We can solve both these problems at once by moving the DotFilter to the files iterator in Dir, rather than at the Dir’s creation. Having the iterator know whether it should display `.` and `..` means it can emit those files first, and because it knows what those files really represent, it can override their file names to actually be those sequences of dots.
This is not a perfect solution: the main casualty is that a File can now be constructed with a name, some metadata, both, or neither. This is currently handled with a bunch of Options, and returns IOResult even without doing any IO operations.
But at least all the tests pass!
I originally thought that the entries . and .. were in *every* directory entry, and exa was already doing something to filter it out. And then... I could find no such code! Turns out, if we want those entries present, we have to insert them ourselves.
This was harder than expected. Because the file filter doesn’t have access to the parent directory path, it can’t “filter” the files vector by inserting the files at the beginning.
Instead, we do it at the iterator level. A directory can be scanned in three different ways depending on what sort of dotfiles, if any, are wanted. At this point, we already have access to the parent directory’s path, so we can just insert them manually. The enum got moved to the dir module because it’s used most there.
