Commit graph

119 commits

Author SHA1 Message Date
Justin Ma
039d0a685a
Fix the document CI error for polars profile command (#14642)
# Description

Fix the docs repo CI build error here:
https://github.com/nushell/nushell.github.io/actions/runs/12425087184/job/34691291790#step:5:18

The doc generated by `make_docs.nu` for `polars profile` command will
make the CI build fail due to the indention error of markdown front
matters. I used to fix it manually before, for the long run, it's better
to fix it from the source code.
2024-12-20 13:47:02 +01:00
Jack Wright
981a000ee8
Added flag --coalesce-columns to allow columns to be coalsced on full joins (#14578)
- fixes #14572

# Description
This allowed columns to be coalesced on full joins with `polars join`,
providing functionality simlar to the old `--outer` join behavior.

# User-Facing Changes
- Provides a new flag `--coalesce-columns` on the `polars join` command
2024-12-17 09:55:42 -08:00
Darren Schroeder
baf86dfb0e
tweak polars join for better cross joins (#14586)
# Description

closes #14585

This PR tries to make `polars join --cross` work better. Example taken
from
https://docs.pola.rs/user-guide/transformations/joins/#cartesian-product

### Before
```nushell
❯ let tokens = [[monopoly_token]; [hat] [shoe] [boat]] | polars into-df
❯ let players = [[name, cash]; [Alice, 78] [Bob, 135]] | polars into-df
❯ $players | polars into-lazy | polars select (polars col name) | polars join --cross $tokens | polars collect
Error: nu::parser::missing_positional

  × Missing required positional argument.
   ╭─[entry #3:1:92]
 1 │ $players | polars into-lazy | polars select (polars col name) | polars join --cross $tokens
   ╰────
  help: Usage: polars join {flags} <other> <left_on> <right_on> . Use `--help` for more information.
```
### After
```nushell
❯ let players = [[name, cash]; [Alice, 78] [Bob, 135]] | polars into-df
❯ let tokens = [[monopoly_token]; [hat] [shoe] [boat]] | polars into-df
❯ $players | polars into-lazy | polars select (polars col name) | polars join --cross $tokens | polars collect
╭─#─┬─name──┬─monopoly_token─╮
│ 0 │ Alice │ hat            │
│ 1 │ Alice │ shoe           │
│ 2 │ Alice │ boat           │
│ 3 │ Bob   │ hat            │
│ 4 │ Bob   │ shoe           │
│ 5 │ Bob   │ boat           │
╰─#─┴─name──┴─monopoly_token─╯
```
Other examples
```nushell
❯ 1..3 | polars into-df | polars join --cross (4..6 | polars into-df)
╭─#─┬─0─┬─0_x─╮
│ 0 │ 1 │   4 │
│ 1 │ 1 │   5 │
│ 2 │ 1 │   6 │
│ 3 │ 2 │   4 │
│ 4 │ 2 │   5 │
│ 5 │ 2 │   6 │
│ 6 │ 3 │   4 │
│ 7 │ 3 │   5 │
│ 8 │ 3 │   6 │
╰─#─┴─0─┴─0_x─╯
❯ 1..3 | each {|x| {x: $x}} | polars into-df | polars join --cross (4..6 | each {|y| {y: $y}} | polars into-df) x y
╭─#─┬─x─┬─y─╮
│ 0 │ 1 │ 4 │
│ 1 │ 1 │ 5 │
│ 2 │ 1 │ 6 │
│ 3 │ 2 │ 4 │
│ 4 │ 2 │ 5 │
│ 5 │ 2 │ 6 │
│ 6 │ 3 │ 4 │
│ 7 │ 3 │ 5 │
│ 8 │ 3 │ 6 │
╰─#─┴─x─┴─y─╯
```
/cc @ayax79 
# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-12-14 21:58:47 -06:00
Jack Wright
219b44a04f
Improve handling of columns with null values (#14588)
Addresses some null handling issues in #6882

# Description

This changes the implementation of guessing a column type when a schema
is not specified.

New behavior:
1. Use the first non-Value::Nothing value type for the columns data type
2. If the value type changes (ignoring Value::Nothing) in subsequent
values, the datatype will be changed to DataType::Object("Value", None)
3. If a column type does not have a value type,
DataType::Object("Value", None) will be assumed.
2024-12-14 18:36:01 -06:00
Jack Wright
81d68cd478
Documentation and error handling around polars with-column --name (#14527)
The `--name` flag of `polars with-column` only works when used with an
eager dataframe. I will not work with lazy dataframes and it will not
work when used with expressions (which forces a conversion to a
lazyframe). This pull request adds better documentation to the flags and
errors messages when used in cases where it will not work.
2024-12-06 05:17:18 -06:00
dependabot[bot]
bf457cd4fc
Bump indexmap from 2.6.0 to 2.7.0 (#14505)
Bumps [indexmap](https://github.com/indexmap-rs/indexmap) from 2.6.0 to
2.7.0.
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/indexmap-rs/indexmap/blob/master/RELEASES.md">indexmap's
changelog</a>.</em></p>
<blockquote>
<h2>2.7.0 (2024-11-30)</h2>
<ul>
<li>Added methods <code>Entry::insert_entry</code> and
<code>VacantEntry::insert_entry</code>, returning
an <code>OccupiedEntry</code> after insertion.</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="539b401151"><code>539b401</code></a>
Merge pull request <a
href="https://redirect.github.com/indexmap-rs/indexmap/issues/361">#361</a>
from cuviper/insert_entry</li>
<li><a
href="998edb12fe"><code>998edb1</code></a>
Release 2.7.0</li>
<li><a
href="2a0ca97417"><code>2a0ca97</code></a>
Add <code>{Entry,VacantEntry}::insert_entry</code></li>
<li><a
href="dceb0f0598"><code>dceb0f0</code></a>
Merge pull request <a
href="https://redirect.github.com/indexmap-rs/indexmap/issues/360">#360</a>
from cuviper/collect_vec_list</li>
<li><a
href="c095322249"><code>c095322</code></a>
ci: downgrade hashbrown for 1.63</li>
<li><a
href="7d8cef8b4b"><code>7d8cef8</code></a>
Use rayon-1.9.0's <code>collect_vec_list</code></li>
<li>See full diff in <a
href="https://github.com/indexmap-rs/indexmap/compare/2.6.0...2.7.0">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=indexmap&package-manager=cargo&previous-version=2.6.0&new-version=2.7.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)


</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-12-04 09:57:05 +08:00
Jack Wright
c63bb81c3e
Convert Filesize to Int (#14491)
# Description
Fixes the conversion of Value::Filesize to Value::Int allowing things
like `ps | polars into-df` to work correctly.
2024-12-03 06:08:41 -06:00
Jack Wright
0172ad8461
Upgrading to polars 0.44 (#14478)
Upgrading to polars 0.44
2024-11-29 19:39:07 -06:00
Jan Klass
6c36bd822c
Fix doc and code comment typos (#14366)
# User-Facing Changes

* Fixes `polars value-counts --column` help text typo
* Fixes `polars agg-groups` help text typo
2024-11-17 19:17:35 +01:00
Jack Wright
9d0f69ac50
Add support for converting polars decimal values to nushell values (#14343)
Adds support for converting from polars decimal type to nushell values.

This fix works by first converting a polars decimal series to an f64
series, then converting to Value::Float

Co-authored-by: Jack Wright <jack.wright@nike.com>
2024-11-15 12:10:38 +08:00
Justin Ma
e6f55da080
Bump to dev version 0.100.1 (#14328) 2024-11-14 10:04:39 +01:00
Justin Ma
c9409a2edb
Bump version to 0.100.0 (#14312)
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx

you can also mention related issues, PRs or discussions!
-->

# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.

Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->

Bump version to `0.100.0`

# User-Facing Changes

The new release `v0.100.0` is coming...
2024-11-12 22:22:38 +02:00
Darren Schroeder
b6eda33438
allow != for polars (#14263)
# Description

This PR fixes a problem where not equal in polars wasn't working with
strings.

## Before
```nushell
let a = ls | polars into-df
$a.type != "dir"
Error: nu:🐚:type_mismatch

  × Type mismatch during operation.
   ╭─[entry #16:1:1]
 1 │ $a.type != "dir"
   · ─┬      ─┬ ──┬──
   ·  │       │   ╰── string
   ·  │       ╰── type mismatch for operator
   ·  ╰── NuDataFrame
   ╰────
```

## After
```nushell
let a = ls | polars into-df
$a.type != "dir"
╭──#──┬─type──╮
│ 0   │ false │
│ 1   │ false │
│ 2   │ false │
...
```

/cc @ayax79 to make sure I did this right.

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-11-06 15:58:22 -08:00
Ian Manske
e87a35104a
Remove as_i64 and as_f64 (#14258)
# Description
Turns out there are duplicate conversion functions: `as_i64` and
`as_f64`. In most cases, these can be replaced with `as_int` and
`as_float`, respectively.
2024-11-05 09:28:56 +01:00
Jack Wright
ae54d05930
Upgrade to polars 0.43 (#14148)
Upgrades the polars plugin to polars version 0.43
2024-10-23 19:14:24 +02:00
dependabot[bot]
e7c4597ad0
Bump uuid from 1.10.0 to 1.11.0 (#14155)
Bumps [uuid](https://github.com/uuid-rs/uuid) from 1.10.0 to 1.11.0.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/uuid-rs/uuid/releases">uuid's
releases</a>.</em></p>
<blockquote>
<h2>1.11.0</h2>
<h2>What's Changed</h2>
<ul>
<li>Upgrade zerocopy to 0.8 by <a
href="https://github.com/yotamofek"><code>@​yotamofek</code></a> in <a
href="https://redirect.github.com/uuid-rs/uuid/pull/771">uuid-rs/uuid#771</a></li>
<li>Prepare for 1.11.0 release by <a
href="https://github.com/KodrAus"><code>@​KodrAus</code></a> in <a
href="https://redirect.github.com/uuid-rs/uuid/pull/772">uuid-rs/uuid#772</a></li>
</ul>
<h2>New Contributors</h2>
<ul>
<li><a href="https://github.com/yotamofek"><code>@​yotamofek</code></a>
made their first contribution in <a
href="https://redirect.github.com/uuid-rs/uuid/pull/771">uuid-rs/uuid#771</a></li>
</ul>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/uuid-rs/uuid/compare/1.10.0...1.11.0">https://github.com/uuid-rs/uuid/compare/1.10.0...1.11.0</a></p>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="4473398413"><code>4473398</code></a>
Merge pull request <a
href="https://redirect.github.com/uuid-rs/uuid/issues/772">#772</a> from
uuid-rs/cargo/1.11.0</li>
<li><a
href="59fbb1e695"><code>59fbb1e</code></a>
prepare for 1.11.0 release</li>
<li><a
href="d9b34e7c93"><code>d9b34e7</code></a>
Merge pull request <a
href="https://redirect.github.com/uuid-rs/uuid/issues/771">#771</a> from
yotamofek/zerocopy_0.8</li>
<li><a
href="14b24206c6"><code>14b2420</code></a>
Upgrade zerocopy to 0.8</li>
<li>See full diff in <a
href="https://github.com/uuid-rs/uuid/compare/1.10.0...1.11.0">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=uuid&package-manager=cargo&previous-version=1.10.0&new-version=1.11.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)


</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-10-23 13:54:16 +08:00
Solomon
b0427ca9ff run ensure_flag_arg_type for short flag values (#14074)
Closes #13654

# User-Facing Changes

- Short flags are now fully type-checked,
  including null and record signatures for literal arguments:

```nushell
def test [-v: record<l: int>] {};
test -v null # error
test -v {l: ""} # error

def test2 [-v: int] {};
let v = ""
test2 -v $v # error
```

- `polars unpivot` `--index`/`--on` and `into value --columns`
now accept `list` values
2024-10-20 23:12:57 +02:00
Jack Wright
f787d272e6 Implemented polars unnest (#14104)
# Description
Provides the ability to decomes struct columns into seperate columns for
each field:
<img width="655" alt="Screenshot 2024-10-16 at 09 57 22"
src="https://github.com/user-attachments/assets/6706bd36-8d38-4365-b58d-ba82f2d5ba9a">

# User-Facing Changes
- provides a new command `polars unnest` for decomposing struct fields
into separate columns.
2024-10-20 23:12:57 +02:00
Stefan Holderbach
f061c9a30e
Bump to 0.99.2 (#14136) 2024-10-20 23:12:41 +02:00
Ian Manske
28b6db115a
Revert PRs for 0.99.1 patch (#14119)
# Description

Temporarily reverts PRs merged after the 0.99.1 bump.
2024-10-18 02:51:14 +00:00
Solomon
2eef42c6b9
run ensure_flag_arg_type for short flag values (#14074)
Closes #13654

# User-Facing Changes

- Short flags are now fully type-checked,
  including null and record signatures for literal arguments:

```nushell
def test [-v: record<l: int>] {};
test -v null # error
test -v {l: ""} # error

def test2 [-v: int] {};
let v = ""
test2 -v $v # error
```

- `polars unpivot` `--index`/`--on` and `into value --columns`
now accept `list` values
2024-10-16 21:25:17 -05:00
Jack Wright
c9d54f821b
Implemented polars unnest (#14104)
# Description
Provides the ability to decomes struct columns into seperate columns for
each field:
<img width="655" alt="Screenshot 2024-10-16 at 09 57 22"
src="https://github.com/user-attachments/assets/6706bd36-8d38-4365-b58d-ba82f2d5ba9a">

# User-Facing Changes
- provides a new command `polars unnest` for decomposing struct fields
into separate columns.
2024-10-16 21:24:14 -05:00
Devyn Cairns
59d6dee3b3
Bump to version 0.99.1 (#14100)
Post-release patch bump.
2024-10-16 21:23:37 -05:00
Jakub Žádník
91ff57faa7
Bump to version 0.99.0 (#14094)
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx

you can also mention related issues, PRs or discussions!
-->

# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.

Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-10-15 22:01:08 +03:00
Jack Wright
2df91e7f92
Removed CustomValue portion of CustomValue type name strings. (#14054)
# Description

This changes the names returned by CustomValue::name() of the various
custom value structs to just say the name of the thing they represent.
For instance "DataFrameCustomValue" is not just "DataFrame".

# User-Facing Changes
- Places such as or errors where NuDataFrameCustomValue would be seen,
now just shows as NuDataFrame.
2024-10-11 06:41:24 -05:00
dependabot[bot]
ba4becc61c
Bump indexmap from 2.5.0 to 2.6.0 (#13983)
Bumps [indexmap](https://github.com/indexmap-rs/indexmap) from 2.5.0 to
2.6.0.
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/indexmap-rs/indexmap/blob/master/RELEASES.md">indexmap's
changelog</a>.</em></p>
<blockquote>
<h2>2.6.0 (2024-10-01)</h2>
<ul>
<li>Implemented <code>Clone</code> for <code>map::IntoIter</code> and
<code>set::IntoIter</code>.</li>
<li>Updated the <code>hashbrown</code> dependency to version 0.15.</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="bf0362ba25"><code>bf0362b</code></a>
Merge pull request <a
href="https://redirect.github.com/indexmap-rs/indexmap/issues/354">#354</a>
from cuviper/release-2.6.0</li>
<li><a
href="bd0b4f7c8c"><code>bd0b4f7</code></a>
Add all release dates</li>
<li><a
href="53400496f4"><code>5340049</code></a>
Release 2.6.0</li>
<li><a
href="7f8022912a"><code>7f80229</code></a>
Merge pull request <a
href="https://redirect.github.com/indexmap-rs/indexmap/issues/343">#343</a>
from cuviper/hash_table</li>
<li><a
href="e577bf2556"><code>e577bf2</code></a>
Use <code>hashbrown::HashTable</code> instead of
<code>RawTable</code></li>
<li><a
href="09b48ec3b3"><code>09b48ec</code></a>
Merge pull request <a
href="https://redirect.github.com/indexmap-rs/indexmap/issues/353">#353</a>
from cuviper/move_index</li>
<li><a
href="267b83d701"><code>267b83d</code></a>
Add an explicit bounds check in <code>move_index</code></li>
<li><a
href="d74a4daffb"><code>d74a4da</code></a>
Merge pull request <a
href="https://redirect.github.com/indexmap-rs/indexmap/issues/349">#349</a>
from waywardmonkeys/improve-doc-formatting</li>
<li><a
href="5b0ed20b87"><code>5b0ed20</code></a>
docs: Improve doc formatting with backticks</li>
<li><a
href="15518f3152"><code>15518f3</code></a>
Merge pull request <a
href="https://redirect.github.com/indexmap-rs/indexmap/issues/348">#348</a>
from cuviper/clone-intoiter</li>
<li>Additional commits viewable in <a
href="https://github.com/indexmap-rs/indexmap/compare/2.5.0...2.6.0">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=indexmap&package-manager=cargo&previous-version=2.5.0&new-version=2.6.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)


</details>

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Jack Wright <jack.wright@disqo.com>
2024-10-08 17:07:21 -07:00
Jack Wright
1d6ac16530
polars into-df struct fix (#13977)
# Description
This fixes an issue with converting to a dataframe when specifying a
struct in the schema. Things like the following now work correctly:
```nushell
 [[foo bar]; [{a: "a_0", b:"b_0"} 1] [{a: "a_1", b: "b_1" } 2]] | polars into-df -s {foo: {a: str, b: str}, bar: u8}
```
2024-10-02 05:59:14 -05:00
Jack Wright
bcaef8959c
Expose flag truncate-ragged-lines in polars open (#13939)
# Description
Introduces a new flag `--truncate-ragged-lines` for `polars open` that
will truncate lines that are longer than the schema.

# User-Facing Changes
- Introduction of the flag `--truncate-ragged-lines` for `polars open`
2024-09-27 06:54:46 -05:00
Jack Wright
5bef81a059
Added command polars len for performing count(*) like operations. (#13941)
# Description
This request exposes the prelude::polars::len expression. It is ended
for doing fast select count(*) like operations:

<img width="626" alt="Screenshot 2024-09-26 at 18 14 45"
src="https://github.com/user-attachments/assets/74285fc6-f99c-46e0-9226-9a7d41738d78">

# User-Facing Changes
- Introduction of the `polars len` command
2024-09-27 06:54:28 -05:00
Jack Wright
28a7461057
Added command polars profile for profiling lazy dataframes (#13904)
# Description
Introduce a new command `polars profile` for profiling lazy dataframes:

<img width="965" alt="Screenshot 2024-09-22 at 23 46 18"
src="https://github.com/user-attachments/assets/11402dd3-8256-43df-a986-64241c15354f">

# User-Facing Changes
- Introduces new command `polars profile`
2024-09-23 07:57:59 -05:00
Jack Wright
2541a712e4
Added polars concat to allow concatenation of multiple dataframes (#13879)
# Description
Provides the ability to concatenate multiple dataframes together

# User-Facing Changes
- Introduces new command `polars concat`
2024-09-23 06:43:43 -05:00
Skyler Hawthorne
5fa9d76500
polars: add binary type support (#13830)
# Description
This adds support for reading and writing binary types in the polars
commands.

The `BinaryOffset` type can be read into a Nushell native `Value` type
no problem, but unfortunately this is a lossy conversion, as there's
no Nushell-native semantic equivalent to the fixed size binary type
in Arrow.

# User-Facing Changes

`polars open` and `polars save` now work with binary types.
2024-09-23 06:28:41 -05:00
Wind
1a081c09de
Bump version to 0.98.1 (#13896) 2024-09-22 12:41:44 +08:00
Devyn Cairns
6e1e824473
Bump version to 0.98.0 (#13865) 2024-09-18 00:48:46 -07:00
Jack Wright
af77bc60e2
Improved null handling when converting from nu -> dataframe. (#13855)
# Description
Fixes: #12726 and #13185

Previously converting columns that contained null caused polars to force
a dtype of object even when using a schema.

Now:
1. When using a schema, the type the schema defines for the column will
always be used.
2. When a schema is not used, the previous type is used when a value is
null.

# User-Facing Changes
- The type defined by the schema we be respected when passing in a null
value `[a]; [null] | polars into-df -s {a: str}` will create a df with
an str dtype column with one null value versus a column of type object.
- *BREAKING CHANGE* If you define a schema, all columns must be in the
schema.
2024-09-16 18:07:13 -05:00
Jack Wright
c535c24d03
catch unwrap on panics with polars collect (#13850)
# Description
This resurrects the work from #12866 and fixes #12732. 

Polars panics for a plethora or reasons. While handling panics is
generally frowned upon, in cases like with `polars collect` a panic
cause a lot of work to be lost. Often you might have multiple dataframes
in memory and you are trying one operation and lose all state.

While it possible the panic can leave things a strange state, it is
pretty unlikely as part of a polars pipeline. Most of the time polars
objects are not manipulating dataframes in memory mutability, but rather
creating a new dataframe the operations being applied. This is always
the case with a lazy pipeline. After the collect call, the original
dataframes are intact still and I haven't observed any side effects.
2024-09-15 07:21:02 -05:00
Jack Wright
fb34a4fc6c
Make polars save return an empty pipeline (#13833)
# Description
In order to be more consistent with it's nu counterpart, `polars save`
now returns an empty pipeline instead of a message of the saved file.

# User-Facing Changes
- `polars save` no longer displays a save message, making it consistent
with `save` behavior.
2024-09-12 06:23:40 -05:00
Jack Wright
8d60c0d35d
Migrating polars commands away from macros, removed custom DataFrame comparison. (#13829)
# Description
This PR:
- Removes the lazy_command, expr_command macros and migrates the
commands that were utilizing them.
- Removes the custom logic in DataFrameValues::is_equals to use the
polars DataFrame version of PartialEq
- Adds examples to commands that previously did not have examples or had
inadequate ones.

NOTE: A lot of examples now have a `polars sort` at the end. This is
needed due to the comparison in the result. The new polars version of
equals cares about the ordering. I removed the custom equals logic as it
causes comparisons to lock up when comparing dataframes that contain a
row that contains a list. I discovered this issue when adding examples
to `polars implode`
2024-09-11 10:33:05 -07:00
Jack Wright
1e64f59220
Added documentation explanation explaining how to select all columns with polars col (#13806)
# Description
Previously there were no examples or explanations that you can use '*'
to select all columns. Updated description and added a new example.
2024-09-08 17:12:03 +00:00
Jack Wright
6c1c7f9509
Added expression support for polars cumulative (#13799)
# Description
Provides the ability to use `polars cumulative` as an expression:

<img width="1266" alt="Screenshot 2024-09-06 at 17 47 15"
src="https://github.com/user-attachments/assets/73c11f79-598c-4efa-bfcd-755e536ead66">

# User-Facing Changes
- `polars cumulative` can now be used as an expression.
2024-09-06 22:03:51 -05:00
Jack Wright
f531cc2058
Polars command reorg (#13798)
# Description
House keeping. Restructures polars modules as discussed in:
https://docs.google.com/spreadsheets/d/1gyA58i_yTXKCJ5DbO_RxBNAlK6S7C1M22ppKwVLZltc/edit?usp=sharing
2024-09-06 13:46:37 -07:00
Jack Wright
dc53c20628
Renamed polars concatenate and added expression support. (#13781)
# Description
In order to be more consistent with the nushell terminology and with
polars expression terminology `polars concatenate` is now `polars
str-join`. `polars str-join` can also be used as expression.

<img width="857" alt="Screenshot 2024-09-04 at 12 41 25"
src="https://github.com/user-attachments/assets/8cc5a0c2-194c-49ec-9fe1-65ec4825414d">


# User-Facing Changes
- `polars concatenate` is now `polars str-join`
- `polars str-join` can be used as an expression
2024-09-05 09:28:34 -07:00
Jack Wright
e7c5f83460
Added expression support for polars str-lengths (#13782)
# Description
Allows `polars str-lengths` to be used as an expression:

<img width="826" alt="Screenshot 2024-09-04 at 13 57 45"
src="https://github.com/user-attachments/assets/b74139e0-e8ba-4910-84c2-cf4be4a084b6">

# User-Facing Changes
- `polars str-lengths` can be used as an expression.
- char length is now the default. Use the --bytes flag to get bytes
length.
2024-09-05 09:26:09 -07:00
Jack Wright
f611196373
Expression support for polars str-slice (#13783)
# Description
Provides expression support for `polars str-slice`:
<img width="893" alt="Screenshot 2024-09-04 at 18 03 05"
src="https://github.com/user-attachments/assets/d8a8a2a7-53cf-4c3a-ae7a-dfdaf48a05ee">

# User-Facing Changes
- `polars str-slice` can now be used as an expression
2024-09-05 09:06:37 -07:00
Jack Wright
63b94dbd28
Added expression support for polars contains (#13769)
# Description
Adds the ability to use `polars contains` as an expression:
<img width="785" alt="Screenshot 2024-09-03 at 14 39 03"
src="https://github.com/user-attachments/assets/35c0d4e5-6bef-4974-a31f-463c7203bd03">


# User-Facing Changes
- `polars contains` can now be used with expressions
2024-09-04 12:19:45 +02:00
Jack Wright
eb0de25d19
Expression support for polars strftime (#13767)
# Description
Allows `polars strftime` to be used as an expression:
<img width="849" alt="Screenshot 2024-09-03 at 13 14 11"
src="https://github.com/user-attachments/assets/2a987fdf-cf00-4c57-b6e1-0b706c594c20">

# User-Facing Changes
- `polars strftime` can now be used as an expression.
2024-09-04 12:19:29 +02:00
dependabot[bot]
348c59b740
Bump indexmap from 2.4.0 to 2.5.0 (#13772)
Bumps [indexmap](https://github.com/indexmap-rs/indexmap) from 2.4.0 to
2.5.0.
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/indexmap-rs/indexmap/blob/master/RELEASES.md">indexmap's
changelog</a>.</em></p>
<blockquote>
<h2>2.5.0</h2>
<ul>
<li>Added an <code>insert_before</code> method to <code>IndexMap</code>
and <code>IndexSet</code>, as an
alternative to <code>shift_insert</code> with different behavior on
existing entries.</li>
<li>Added <code>first_entry</code> and <code>last_entry</code> methods
to <code>IndexMap</code>.</li>
<li>Added <code>From</code> implementations between
<code>IndexedEntry</code> and <code>OccupiedEntry</code>.</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="48ed49017c"><code>48ed490</code></a>
Release 2.5.0</li>
<li><a
href="139d7addfb"><code>139d7ad</code></a>
Merge pull request <a
href="https://redirect.github.com/indexmap-rs/indexmap/issues/340">#340</a>
from cuviper/insert-bounds</li>
<li><a
href="1d9b5e3d03"><code>1d9b5e3</code></a>
Add doc examples for <code>insert_before</code> and
<code>shift_insert</code></li>
<li><a
href="8ca01b0df7"><code>8ca01b0</code></a>
Use <code>insert_before</code> for &quot;new&quot; entries in
<code>insert_sorted</code></li>
<li><a
href="7224def010"><code>7224def</code></a>
Add <code>insert_before</code> as an alternate to
<code>shift_insert</code></li>
<li><a
href="0247a1555d"><code>0247a15</code></a>
Document and assert index bounds in <code>shift_insert</code></li>
<li><a
href="922c6ad1af"><code>922c6ad</code></a>
Update the CI badge</li>
<li><a
href="e482e1768a"><code>e482e17</code></a>
Merge pull request <a
href="https://redirect.github.com/indexmap-rs/indexmap/issues/342">#342</a>
from cuviper/btree-like</li>
<li><a
href="b63e4a1556"><code>b63e4a1</code></a>
Merge pull request <a
href="https://redirect.github.com/indexmap-rs/indexmap/issues/341">#341</a>
from cuviper/from-entry</li>
<li><a
href="264e5b7304"><code>264e5b7</code></a>
Add doc aliases like <code>BTreeMap</code>/<code>BTreeSet</code></li>
<li>Additional commits viewable in <a
href="https://github.com/indexmap-rs/indexmap/compare/2.4.0...2.5.0">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=indexmap&package-manager=cargo&previous-version=2.4.0&new-version=2.5.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)


</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-09-04 10:02:01 +08:00
Jack Wright
0119534f61
Expression support polars replace and polars replace-all (#13726)
# Description
Adds the ability for `polars replace` and `polars replace-all` to work
as expressions.

# User-Facing Changes
- `polars replace` can be used with polars expressions
- `polars replace-all` can be used with polars expressions
2024-08-29 13:59:44 -07:00
Jack Wright
644bebf4c6
Expression support for polars uppercase and polars lowercase (#13724) 2024-08-28 14:08:16 -07:00
Jack Wright
a39e94de8a
Added polars commands for converting string columns to integer and decimal columns (#13711)
# Description
Introduces two new polars commands for converting string columns to
decimal and integer columns:

<img width="740" alt="Screenshot 2024-08-27 at 15 32 28"
src="https://github.com/user-attachments/assets/f9573b6e-48f6-4bbf-8782-39ffb95eb934">

<img width="720" alt="Screenshot 2024-08-27 at 15 33 46"
src="https://github.com/user-attachments/assets/90a66bb5-fa78-4ed3-8b2b-ae05cddd2f3a">

# User-Facing Changes
- Addition of the `polars integer` command
- Addition of the `polars decimal` command
2024-08-28 07:54:31 -05:00