modin.pandas.concat

modin.pandas.concat(objs: Iterable[pd.DataFrame | Series] | Mapping[Hashable, pd.DataFrame | Series], axis: Axis = 0, join: str = 'outer', ignore_index: bool = False, keys: Sequence[Hashable] = None, levels: list[Sequence[Hashable]] = None, names: list[Hashable] = None, verify_integrity: bool = False, sort: bool = False, copy: bool = True) → pd.DataFrame | Series[source] (https://github.com/snowflakedb/snowpark-python/blob/v1.26.0/snowpark-python/src/snowflake/snowpark/modin/plugin/extensions/general_overrides.py#L748-L881)

Concatenate pandas objects along a particular axis.

Allows optional set logic along the other axes.

Can also add a layer of hierarchical indexing on the concatenation axis, which may be useful if the labels are the same (or overlapping) on the passed axis number.

Parameters:

  • objs (a sequence or mapping of Series or DataFrame objects) – If a mapping is passed, the sorted keys will be used as the keys argument, unless it is passed, in which case the values will be selected (see below). Any None objects will be dropped silently unless they are all None in which case a ValueError will be raised.

  • axis ({0/'index', 1/'columns'}, default 0) – The axis to concatenate along.

  • join ({'inner', 'outer'}, default 'outer') – How to handle indexes on other axis (or axes).

  • ignore_index (bool, default False) – If True, do not use the index values along the concatenation axis. The resulting axis will be labeled 0, …, n - 1. This is useful if you are concatenating objects where the concatenation axis does not have meaningful indexing information. Note the index values on the other axes are still respected in the join.

  • keys (sequence, default None) – If multiple levels passed, should contain tuples. Construct hierarchical index using the passed keys as the outermost level.

  • levels (list of sequences, default None) – Specific levels (unique values) to use for constructing a MultiIndex. Otherwise they will be inferred from the keys. Snowpark pandas does not support ‘levels’ argument.

  • names (list, default None) – Names for the levels in the resulting hierarchical index.

  • verify_integrity (bool, default False) – Check whether the new concatenated axis contains duplicates. Snowpark pandas does not support distributed computation of concat when ‘verify_integrity’ is True.

  • sort (bool, default False) – Sort non-concatenation axis if it is not already aligned.

  • copy (bool, default True) – If False, do not copy data unnecessarily. This argument is ignored in Snowpark pandas.

Returns:

When concatenating all Snowpark pandas Series along the index (axis=0), a Snowpark pandas Series is returned. When objs contains at least one Snowpark pandas DataFrame, a Snowpark pandas DataFrame is returned. When concatenating along the columns (axis=1), a Snowpark pandas DataFrame is returned.

Return type:

object, type of objs

See also

DataFrame.join

Join DataFrames using indexes.

DataFrame.merge

Merge DataFrames by indexes or columns.

Notes

The keys, levels, and names arguments are all optional.

It is not recommended to build DataFrames by adding single rows in a for loop. Build a list of rows and make a DataFrame in a single concat.

Examples

Combine two Series.

>>> s1 = pd.Series(['a', 'b'])
>>> s2 = pd.Series(['c', 'd'])
>>> pd.concat([s1, s2])
0    a
1    b
0    c
1    d
dtype: object
Copy

Clear the existing index and reset it in the result by setting the ignore_index option to True.

>>> pd.concat([s1, s2], ignore_index=True)
0    a
1    b
2    c
3    d
dtype: object
Copy

Add a hierarchical index at the outermost level of the data with the keys option.

>>> pd.concat([s1, s2], keys=['s1', 's2'])
s1  0    a
    1    b
s2  0    c
    1    d
dtype: object
Copy

Label the index keys you create with the names option.

>>> pd.concat([s1, s2], keys=['s1', 's2'],
...           names=['Series name', 'Row ID'])
Series name  Row ID
s1           0         a
             1         b
s2           0         c
             1         d
dtype: object
Copy

Combine two DataFrame objects with identical columns.

>>> df1 = pd.DataFrame([['a', 1], ['b', 2]],
...                    columns=['letter', 'number'])
>>> df1
  letter  number
0      a       1
1      b       2
>>> df2 = pd.DataFrame([['c', 3], ['d', 4]],
...                    columns=['letter', 'number'])
>>> df2
  letter  number
0      c       3
1      d       4
>>> pd.concat([df1, df2])
  letter  number
0      a       1
1      b       2
0      c       3
1      d       4
Copy

Combine DataFrame objects with overlapping columns and return everything. Columns outside the intersection will be filled with NaN values.

>>> df3 = pd.DataFrame([['c', 3, 'cat'], ['d', 4, 'dog']],
...                    columns=['letter', 'number', 'animal'])
>>> df3
  letter  number animal
0      c       3    cat
1      d       4    dog
>>> pd.concat([df1, df3], sort=False)
  letter  number animal
0      a       1   None
1      b       2   None
0      c       3    cat
1      d       4    dog
Copy

Combine DataFrame objects with overlapping columns and return only those that are shared by passing inner to the join keyword argument.

>>> pd.concat([df1, df3], join="inner")
  letter  number
0      a       1
1      b       2
0      c       3
1      d       4
Copy

Combine DataFrame objects horizontally along the x axis by passing in axis=1.

>>> df4 = pd.DataFrame([['bird', 'polly'], ['monkey', 'george']],
...                    columns=['animal', 'name'])
>>> pd.concat([df1, df4], axis=1)
  letter  number  animal    name
0      a       1    bird   polly
1      b       2  monkey  george
Copy

Combining series horizontally creates a DataFrame. Missing names are replaced with numeric values.

>>> pd.concat([s1, s2], axis=1)
   0  1
0  a  c
1  b  d
Copy

When combining objects horizoantally ignore_index=True will clear the existing column names and reset it in the result.

>>> pd.concat([df1, df4], axis=1, ignore_index=True)
   0  1       2       3
0  a  1    bird   polly
1  b  2  monkey  george
Copy

When combining objects horizontally, add a hierarchical column index at the outermost level of the column labels with the keys option.

>>> pd.concat([df1, df4], axis=1, keys=['x', 'y']) 
       x              y
  letter number  animal    name
0      a      1    bird   polly
1      b      2  monkey  george
Copy

Concatenatiing series horizontally with keys.

>>> pd.concat([s1, s2], axis=1, keys=['x', 'y'])
   x  y
0  a  c
1  b  d
Copy

When combining objects horizontally, how='inner' to keep only overalpping index values.

>>> df5 = pd.DataFrame([['a', 1], ['b', 2]],
...                    columns=['letter', 'number'],
...                    index=[1, 2])
>>> df5
  letter  number
1      a       1
2      b       2
>>> pd.concat([df1, df5], axis=1, join='inner')
  letter  number letter  number
1      b       2      a       1
Copy

Prevent the result from including duplicate index values with the verify_integrity option.

>>> df5 = pd.DataFrame([1], index=['a'])
>>> df5
   0
a  1
>>> df6 = pd.DataFrame([2], index=['a'])
>>> df6
   0
a  2
>>> pd.concat([df5, df6], verify_integrity=True)
Traceback (most recent call last):
    ...
ValueError: Indexes have overlapping values: Index(['a'], dtype='object')
Copy

Append a single row to the end of a DataFrame object.

>>> df7 = pd.DataFrame({'a': 1, 'b': 2}, index=[0])
>>> df7
   a  b
0  1  2
>>> new_row = pd.DataFrame({'a': 3, 'b': 4}, index=[0])
>>> new_row
   a  b
0  3  4
>>> pd.concat([df7, new_row], ignore_index=True)
   a  b
0  1  2
1  3  4
Copy
Language: English