decimal vectors now convert to and from 'Arrow' decimal arrays directly.
as_decimal() gained methods for Array and ChunkedArray. An Arrow
decimal column of any width converts exactly, taking its scale from the
Arrow type: Arrow's own decimal-to-string cast produces this package's
canonical storage form once the exponent letter is folded to lowercase, so
the strings need no re-parsing, and a property test guards that invariant.
For 1,000,000 decimal128() values that takes 0.27s against
0.78s for casting to string and calling decimal(). Arrow integer
columns convert exactly at every width, including int64 and uint64
values a double cannot hold. Other Arrow types convert through the ordinary
as_decimal() rules for the equivalent R vector.
A decimal column now becomes an Arrow decimal field in
arrow::arrow_table(), arrow::write_parquet() and
arrow::write_dataset(). The field is an Arrow extension type whose storage
is a real decimal128() or decimal256(), so Spark, DuckDB, pandas and
other readers see a plain decimal column, while in R it returns as a
decimal vector on every read path: as.data.frame(),
arrow::read_parquet() and dplyr::collect() included. New
arrow_decimal_type() pins a precision and scale for a column that will be
appended to. Infinities and NaNs, which no Arrow decimal can represent, now
raise an error on conversion instead of round-tripping through the 'vctrs'
extension type; keep such a column as a string if you need them.
Arrow's compute engine does not operate on extension columns. For arrow-side
arithmetic or filtering on a decimal column, write a plain field by passing
a plain Arrow decimal type to arrow::as_arrow_array() or by setting
options(decimal.arrow_extension = FALSE). A plain field comes back from
arrow's own conversion as a double wearing the decimal class, because
arrow reapplies the column's recorded R attributes; format() and
as.character() now refuse such an object rather than print rounded
values. New arrow_as_data_frame() converts an Arrow Table or
RecordBatch to a data frame, reading plain decimal fields, including
those in files written by other systems, as decimal vectors, and leaving
every other column to arrow.
Fixed memory leaks on the error paths of the native kernels. An R error
raised while an mpd_t was allocated abandoned it, because R's error
handling unwinds past the code that would have freed it. Parsing an invalid
decimal string leaked every handle the operation held, and an allocation
failure part-way through setting up an operation leaked the handles already
allocated. An R allocation failure while storing a result string leaked the
formatted text too; that text is now released under R_UnwindProtect().
summary() now works on decimal vectors. It reports the same six
statistics as [summary.default()] plus an NA's count, computed in decimal
arithmetic and returned as a decimal vector, so figures that a double would
round are preserved. Quartiles use the type 7 definition, matching
stats::quantile().
Fixed is.na(), is.nan(), is.infinite(), is_qnan() and is_snan()
returning malformed logical vectors. They forwarded mpdecimal's flag bits --
4 for NaN, 8 for sNaN, 2 for Infinity -- instead of TRUE. The results
printed correctly and compared equal with ==, but which() found no
matching elements, sum() over-counted, and identical() was false against
the obvious expectation.
DESCRIPTION now declares URL and BugReports, so the CRAN page links to
the source repository and the issue tracker.
Building a decimal vector from character is substantially faster. The
fractional-digit scan that decimal() performs on every element moved from R
to C, which for 100,000 values takes construction from 1.13s to 0.05s when
scale is supplied (about 22x) and from 2.28s to 0.05s when it is inferred
(about 45x). Inferring the scale now costs the same as supplying it. This
matters most when loading decimal data from 'Arrow' or a database, where the
values arrive as a character vector in one call.
New vignette vignette("arrow-decimal-types"), covering lossless conversion
between decimal vectors and Arrow's decimal128() / decimal256() types
in both directions, including chunked arrays, Parquet columns, and the cases
Arrow decimals cannot represent.
format() on a decimal vector now accepts the extra arguments that table
printers pass to every column, such as digits, na.encode and justify
from format.data.frame(), timezone from 'data.table' and trim from
knitr::kable(), so decimal columns print inside a base data.frame, a
data.table and a kable() table. The arguments are ignored rather than
honored, so an exact value is never silently rounded for display. Arguments
that would change how a number is written, such as nsmall, scientific
and big.mark, are an error.
Fixed rbind() on data frames with decimal columns, which always failed
with "Can't assign to elements past the end". Assigning past the end of a
decimal vector now grows it with missing values, as it does a base vector,
and binding columns of different scales takes the finer one. x[] <- value
now replaces every element instead of failing.
match(), %in% and base merge() now compare decimal values rather
than their stored text, so 2.5 matches 2.50, as == already said.
Whole numbers still match integers and strings: decimal("20") %in% 20L
stays TRUE.
New section in vignette("decimal-values") on decimal columns in a
'data.table': what works, and workarounds for the operations data.table runs
on the stored text instead of the values, such as sorting and grouped
min() and max().