Published in Towards Data Science·PinnedUsing Higher-Order Containers to Efficiently Process 7,163 (or More) DataFramesIntroducing the StaticFrame Bus, Batch, Quilt, and Yarn — DataFrame processing routines commonly work with collections of tables. Examples of such collections include a multi-year dataset with a single table per year, historical stock data with a table per ticker, or data from multiple sheets in an XLSX file. This article introduces novel “higher-order” containers for working with such…Dataframes14 min readDataframes14 min read
Published in Towards Data Science·20 hours agoThe Performance Advantage of No-Copy DataFrame OperationsHow StaticFrame can outperform Pandas by embracing NumPy array views — A NumPy array is a Python object that stores data in a contiguous C-array buffer. The excellent performance of these arrays comes not only from this compact representation, but also from the ability of arrays to share “views” of that buffer among many arrays. NumPy makes frequent use of “no-copy”…Python7 min readPython7 min read
Published in Towards Data Science·Jul 20One Fill Value Is Not Enough: Preserving Columnar Types When Reindexing DataFramesWhen working with DataFrames, reindexing is common. When a DataFrame is reindexed, an old index (and its associated values) is conformed to a new index, potentially reordering, contracting, or expanding the rows or columns. When a reindex expands a DataFrame, new values are needed to fill the newly created rows…Dataframes6 min readDataframes6 min read
Published in Towards Data Science·Sep 13, 2020Member-onlyTen Reasons to Use StaticFrame Instead of PandasCreating more maintainable, less error-prone Python when working with DataFrames — If you work with data in Python, you probably use Pandas. Pandas provides nearly instant gratification: sophisticated data processing routines can be implemented in a few lines of code. However, if you have used Pandas on large projects over many years, you may have had some challenges. Complex Pandas applications…Python15 min readPython15 min read
Aug 18, 2019Boring Indices & Where to Find Them: The Auto-Incremented Integer Index in StaticFrameThis article is part of a series exploring the features and design of StaticFrame, a Python package that offers data structures similar to the Pandas DataFrame and Series, but with an immutable data model. This article demonstrates how StaticFrame exposes functionality for creating the most boring index object: the auto-incremented…Python6 min readPython6 min read
