o 5cF;@szdZddlmZddlmZddlZddlmZmZddl Z ddl m Z ddl Z ddl mZmZmZmZmZmZmZmZmZmZmZmZmZddlZddlZddlmZdd l m!Z!m"Z"ddl#m$m%Z&dd l'm(Z(m)Z)m*Z*m+Z+m,Z,m-Z-m.Z.m/Z/dd l0m1Z2dd l3m4Z4m5Z5dd l6m7Z7m8Z8m9Z9m:Z:ddl;mZ>ddl?m@Z@mAZAmBZBmCZCmDZDmEZEmFZFmGZGmHZHddlImJZJmKZKddlLmMZMddlNmOZOddlPmQmRZRddlSmTZTmUZUmVZVmWZWddlXmYZYmZZZddl[mQm\Z]ddl^m_Z_ddl`maZaddlbmcZcmdZdmeZeddlfmgZgmhZhddlimjZjmkZkmlZlmmZmddlnmoZoddlpmQmqZqddlrmsZsddltmuZuddlvmwZwmxZxerIddlymzZzm{Z{m|Z|d Z}d!d"d#d$Z~d%Zd&Zd'Zd(ZeGd)d*d*eYZeeeeeegefeeegefeeeffZGd+d,d,eYeZe*egZed-ead.ZGd/d0d0ee*Ze:e       1 1 1 2 2 2 1dSdTdEdFZdUdLdMZdVdQdRZdS)Wa Provide the groupby split-apply-combine paradigm. Define the GroupBy class providing the base-class of operations. The SeriesGroupBy and DataFrameGroupBy sub-class (defined in pandas.core.groupby.generic) expose these user-facing objects to provide specific functionality. ) annotations)contextmanagerN)partialwraps)dedent) TYPE_CHECKINGCallableHashableIterableIteratorListLiteralMappingSequenceTypeVarUnioncastfinal)option_context) Timestamplib) ArrayLike IndexLabelNDFrameTPositionalIndexer RandomStateScalarTnpt)function)AbstractMethodError DataError)Appender Substitutioncache_readonlydoc)find_stack_level)ensure_dtype_can_hold_na) is_bool_dtypeis_datetime64_dtypeis_float_dtype is_integeris_integer_dtypeis_numeric_dtypeis_object_dtype is_scalaris_timedelta64_dtype)isnanotna)nanops)executor)BaseMaskedArray BooleanArray CategoricalExtensionArray) PandasObjectSelectionMixin) DataFrame)NDFrame)basenumba_ops)GroupByIndexingMixinGroupByNthSelector)CategoricalIndexIndex MultiIndex RangeIndex)ensure_block_shape)Series)get_group_index_sorter)get_jit_argumentsmaybe_use_numba)ExpandingGroupbyExponentialMovingWindowGroupbyRollingGroupbyz See Also -------- Series.%(name)s : Apply a function %(name)s to a Series. DataFrame.%(name)s : Apply a function %(name)s to each row or column of a DataFrame. a] Apply function ``func`` group-wise and combine the results together. The function passed to ``apply`` must take a {input} as its first argument and return a DataFrame, Series or scalar. ``apply`` will then take care of combining the results back together into a single dataframe or series. ``apply`` is therefore a highly flexible grouping method. While ``apply`` is a very flexible method, its downside is that using it can be quite a bit slower than using more specific methods like ``agg`` or ``transform``. Pandas offers a wide range of method that will be much faster than using ``apply`` for their specific purposes, so try to use them before reaching for ``apply``. Parameters ---------- func : callable A callable that takes a {input} as its first argument, and returns a dataframe, a series or a scalar. In addition the callable may take positional and keyword arguments. args, kwargs : tuple and dict Optional positional and keyword arguments to pass to ``func``. Returns ------- applied : Series or DataFrame See Also -------- pipe : Apply function to the full GroupBy object instead of to each group. aggregate : Apply aggregate function to the GroupBy object. transform : Apply function column-by-column to the GroupBy object. Series.apply : Apply a function to a Series. DataFrame.apply : Apply a function to each row or column of a DataFrame. Notes ----- .. versionchanged:: 1.3.0 The resulting dtype will reflect the return value of the passed ``func``, see the examples below. Functions that mutate the passed object can produce unexpected behavior or errors and are not supported. See :ref:`gotchas.udf-mutation` for more details. Examples -------- {examples} a& >>> df = pd.DataFrame({'A': 'a a b'.split(), ... 'B': [1,2,3], ... 'C': [4,6,5]}) >>> g1 = df.groupby('A', group_keys=False) >>> g2 = df.groupby('A', group_keys=True) Notice that ``g1`` have ``g2`` have two groups, ``a`` and ``b``, and only differ in their ``group_keys`` argument. Calling `apply` in various ways, we can get different grouping results: Example 1: below the function passed to `apply` takes a DataFrame as its argument and returns a DataFrame. `apply` combines the result for each group together into a new DataFrame: >>> g1[['B', 'C']].apply(lambda x: x / x.sum()) B C 0 0.333333 0.4 1 0.666667 0.6 2 1.000000 1.0 In the above, the groups are not part of the index. We can have them included by using ``g2`` where ``group_keys=True``: >>> g2[['B', 'C']].apply(lambda x: x / x.sum()) B C A a 0 0.333333 0.4 1 0.666667 0.6 b 2 1.000000 1.0 Example 2: The function passed to `apply` takes a DataFrame as its argument and returns a Series. `apply` combines the result for each group together into a new DataFrame. .. versionchanged:: 1.3.0 The resulting dtype will reflect the return value of the passed ``func``. >>> g1[['B', 'C']].apply(lambda x: x.astype(float).max() - x.min()) B C A a 1.0 2.0 b 0.0 0.0 >>> g2[['B', 'C']].apply(lambda x: x.astype(float).max() - x.min()) B C A a 1.0 2.0 b 0.0 0.0 The ``group_keys`` argument has no effect here because the result is not like-indexed (i.e. :ref:`a transform `) when compared to the input. Example 3: The function passed to `apply` takes a DataFrame as its argument and returns a scalar. `apply` combines the result for each group together into a Series, including setting the index as appropriate: >>> g1.apply(lambda x: x.C.max() - x.B.min()) A a 5 b 2 dtype: int64a >>> s = pd.Series([0, 1, 2], index='a a b'.split()) >>> g1 = s.groupby(s.index, group_keys=False) >>> g2 = s.groupby(s.index, group_keys=True) From ``s`` above we can see that ``g`` has two groups, ``a`` and ``b``. Notice that ``g1`` have ``g2`` have two groups, ``a`` and ``b``, and only differ in their ``group_keys`` argument. Calling `apply` in various ways, we can get different grouping results: Example 1: The function passed to `apply` takes a Series as its argument and returns a Series. `apply` combines the result for each group together into a new Series. .. versionchanged:: 1.3.0 The resulting dtype will reflect the return value of the passed ``func``. >>> g1.apply(lambda x: x*2 if x.name == 'a' else x/2) a 0.0 a 2.0 b 1.0 dtype: float64 In the above, the groups are not part of the index. We can have them included by using ``g2`` where ``group_keys=True``: >>> g2.apply(lambda x: x*2 if x.name == 'a' else x/2) a a 0.0 a 2.0 b b 1.0 dtype: float64 Example 2: The function passed to `apply` takes a Series as its argument and returns a scalar. `apply` combines the result for each group together into a Series, including setting the index as appropriate: >>> g1.apply(lambda x: x.max() - x.min()) a 1 b 0 dtype: int64 The ``group_keys`` argument has no effect here because the result is not like-indexed (i.e. :ref:`a transform `) when compared to the input. >>> g2.apply(lambda x: x.max() - x.min()) a 1 b 0 dtype: int64)templatedataframe_examplesseries_examplesa Compute {fname} of group values. Parameters ---------- numeric_only : bool, default {no} Include only float, int, boolean columns. If None, will attempt to use everything, then use only numeric data. min_count : int, default {mc} The required number of valid values to perform the operation. If fewer than ``min_count`` non-NA values are present the result will be NA. Returns ------- Series or DataFrame Computed {fname} of values within each group. aO Apply a ``func`` with arguments to this %(klass)s object and return its result. Use `.pipe` when you want to improve readability by chaining together functions that expect Series, DataFrames, GroupBy or Resampler objects. Instead of writing >>> h(g(f(df.groupby('group')), arg1=a), arg2=b, arg3=c) # doctest: +SKIP You can write >>> (df.groupby('group') ... .pipe(f) ... .pipe(g, arg1=a) ... .pipe(h, arg2=b, arg3=c)) # doctest: +SKIP which is much more readable. Parameters ---------- func : callable or tuple of (callable, str) Function to apply to this %(klass)s object or, alternatively, a `(callable, data_keyword)` tuple where `data_keyword` is a string indicating the keyword of `callable` that expects the %(klass)s object. args : iterable, optional Positional arguments passed into `func`. kwargs : dict, optional A dictionary of keyword arguments passed into `func`. Returns ------- object : the return type of `func`. See Also -------- Series.pipe : Apply a function with arguments to a series. DataFrame.pipe: Apply a function with arguments to a dataframe. apply : Apply function to each group instead of to the full %(klass)s object. Notes ----- See more `here `_ Examples -------- %(examples)s ac Call function producing a same-indexed %(klass)s on each group. Returns a %(klass)s having the same indexes as the original object filled with the transformed values. Parameters ---------- f : function Function to apply to each group. See the Notes section below for requirements. Can also accept a Numba JIT function with ``engine='numba'`` specified. If the ``'numba'`` engine is chosen, the function must be a user defined function with ``values`` and ``index`` as the first and second arguments respectively in the function signature. Each group's index will be passed to the user defined function and optionally available for use. .. versionchanged:: 1.1.0 *args Positional arguments to pass to func. engine : str, default None * ``'cython'`` : Runs the function through C-extensions from cython. * ``'numba'`` : Runs the function through JIT compiled code from numba. * ``None`` : Defaults to ``'cython'`` or the global setting ``compute.use_numba`` .. versionadded:: 1.1.0 engine_kwargs : dict, default None * For ``'cython'`` engine, there are no accepted ``engine_kwargs`` * For ``'numba'`` engine, the engine can accept ``nopython``, ``nogil`` and ``parallel`` dictionary keys. The values must either be ``True`` or ``False``. The default ``engine_kwargs`` for the ``'numba'`` engine is ``{'nopython': True, 'nogil': False, 'parallel': False}`` and will be applied to the function .. versionadded:: 1.1.0 **kwargs Keyword arguments to be passed into func. Returns ------- %(klass)s See Also -------- %(klass)s.groupby.apply : Apply function ``func`` group-wise and combine the results together. %(klass)s.groupby.aggregate : Aggregate using one or more operations over the specified axis. %(klass)s.transform : Call ``func`` on self producing a %(klass)s with the same axis shape as self. Notes ----- Each group is endowed the attribute 'name' in case you need to know which group you are working on. The current implementation imposes three requirements on f: * f must return a value that either has the same shape as the input subframe or can be broadcast to the shape of the input subframe. For example, if `f` returns a scalar it will be broadcast to have the same shape as the input subframe. * if this is a DataFrame, f must support application column-by-column in the subframe. If f also supports application to the entire subframe, then a fast path is used starting from the second chunk. * f must not mutate groups. Mutation is not supported and may produce unexpected results. See :ref:`gotchas.udf-mutation` for more details. When using ``engine='numba'``, there will be no "fall back" behavior internally. The group data and group index will be passed as numpy arrays to the JITed user defined function, and no alternative execution attempts will be tried. .. versionchanged:: 1.3.0 The resulting dtype will reflect the return value of the passed ``func``, see the examples below. .. deprecated:: 1.5.0 When using ``.transform`` on a grouped DataFrame and the transformation function returns a DataFrame, currently pandas does not align the result's index with the input's index. This behavior is deprecated and alignment will be performed in a future version of pandas. You can apply ``.to_numpy()`` to the result of the transformation function to avoid alignment. Examples -------- >>> df = pd.DataFrame({'A' : ['foo', 'bar', 'foo', 'bar', ... 'foo', 'bar'], ... 'B' : ['one', 'one', 'two', 'three', ... 'two', 'two'], ... 'C' : [1, 5, 5, 2, 5, 5], ... 'D' : [2.0, 5., 8., 1., 2., 9.]}) >>> grouped = df.groupby('A')[['C', 'D']] >>> grouped.transform(lambda x: (x - x.mean()) / x.std()) C D 0 -1.154701 -0.577350 1 0.577350 0.000000 2 0.577350 1.154701 3 -1.154701 -1.000000 4 0.577350 -0.577350 5 0.577350 1.000000 Broadcast result of the transformation >>> grouped.transform(lambda x: x.max() - x.min()) C D 0 4.0 6.0 1 3.0 8.0 2 4.0 6.0 3 3.0 8.0 4 4.0 6.0 5 3.0 8.0 .. versionchanged:: 1.3.0 The resulting dtype will reflect the return value of the passed ``func``, for example: >>> grouped.transform(lambda x: x.astype(int).max()) C D 0 5 8 1 5 9 2 5 8 3 5 9 4 5 8 5 5 9 a Aggregate using one or more operations over the specified axis. Parameters ---------- func : function, str, list or dict Function to use for aggregating the data. If a function, must either work when passed a {klass} or when passed to {klass}.apply. Accepted combinations are: - function - string function name - list of functions and/or function names, e.g. ``[np.sum, 'mean']`` - dict of axis labels -> functions, function names or list of such. Can also accept a Numba JIT function with ``engine='numba'`` specified. Only passing a single function is supported with this engine. If the ``'numba'`` engine is chosen, the function must be a user defined function with ``values`` and ``index`` as the first and second arguments respectively in the function signature. Each group's index will be passed to the user defined function and optionally available for use. .. versionchanged:: 1.1.0 *args Positional arguments to pass to func. engine : str, default None * ``'cython'`` : Runs the function through C-extensions from cython. * ``'numba'`` : Runs the function through JIT compiled code from numba. * ``None`` : Defaults to ``'cython'`` or globally setting ``compute.use_numba`` .. versionadded:: 1.1.0 engine_kwargs : dict, default None * For ``'cython'`` engine, there are no accepted ``engine_kwargs`` * For ``'numba'`` engine, the engine can accept ``nopython``, ``nogil`` and ``parallel`` dictionary keys. The values must either be ``True`` or ``False``. The default ``engine_kwargs`` for the ``'numba'`` engine is ``{{'nopython': True, 'nogil': False, 'parallel': False}}`` and will be applied to the function .. versionadded:: 1.1.0 **kwargs Keyword arguments to be passed into func. Returns ------- {klass} See Also -------- {klass}.groupby.apply : Apply function func group-wise and combine the results together. {klass}.groupby.transform : Aggregate using one or more operations over the specified axis. {klass}.aggregate : Transforms the Series on each group based on the given function. Notes ----- When using ``engine='numba'``, there will be no "fall back" behavior internally. The group data and group index will be passed as numpy arrays to the JITed user defined function, and no alternative execution attempts will be tried. Functions that mutate the passed object can produce unexpected behavior or errors and are not supported. See :ref:`gotchas.udf-mutation` for more details. .. versionchanged:: 1.3.0 The resulting dtype will reflect the return value of the passed ``func``, see the examples below. {examples}c@s,eZdZdZdddZdd Zdd d ZdS) GroupByPlotzE Class implementing the .plot attribute for groupby objects. groupbyGroupByreturnNonecCs ||_dSN)_groupby)selfrRrYQ/var/www/html/gps/gps/lib/python3.10/site-packages/pandas/core/groupby/groupby.py__init__Vs zGroupByPlot.__init__cs fdd}d|_|j|S)Ncs|jiSrV)plotrXargskwargsrYrZfZszGroupByPlot.__call__..fr\)__name__rWapply)rXr_r`rarYr^rZ__call__Ys zGroupByPlot.__call__namestrcsfdd}|S)Ncsfdd}j|S)Ncst|jiSrV)getattrr\r])r_r`rerYrZrabsz0GroupByPlot.__getattr__..attr..f)rWrc)r_r`rarerXr^rZattras z%GroupByPlot.__getattr__..attrrY)rXrerirYrhrZ __getattr__`szGroupByPlot.__getattr__N)rRrSrTrU)rerf)rb __module__ __qualname____doc__r[rdrjrYrYrYrZrQPs  rQc@s*eZdZUdZded<eZded<ejhdBZded<d ed <dZ d ed <d ed<e d4ddZ e d5ddZ e e d6ddZe e d4ddZe e d7ddZe ddZe dd Ze ed!d"Ze d8d$d%Zed&ed'd(eed9d,d-Ze eZe d:d;d/d0Ze d objaxiskeyssortleveldropnagroupermutatedsqueezeas_indexobserved exclusions group_keysintrsops.BaseGrouperrx_KeysArgType | Nonertbool | lib.NoDefaultr~rTcCs t|jSrV)lengroupsr]rYrYrZ__len__s zBaseGroupBy.__len__rfcC t|SrV)object__repr__r]rYrYrZrs zBaseGroupBy.__repr__dict[Hashable, np.ndarray]cC|jjS)z4 Dict {group name -> group labels}. )rxrr]rYrYrZrzBaseGroupBy.groupscCrrV)rxngroupsr]rYrYrZrzBaseGroupBy.ngroups$dict[Hashable, npt.NDArray[np.intp]]cCr)z5 Dict {group name -> group indices}. )rxindicesr]rYrYrZrrzBaseGroupBy.indicesc sddt|dkr gStjdkrttj}nd}|d}t|trjt|ts1d}t|t|t|ksWz fdd|DWStyV}zd}t||d}~wwfd d|Dfd d |D}n |fd d |D}fd d|DS)zd Safe get multiple indices, translate keys for datelike to underlying repr. cSs0t|tjr ddSt|tjrddSddS)NcSt|SrV)rkeyrYrYrZzABaseGroupBy._get_indices..get_converter..cSs t|jSrV)rasm8rrYrYrZr cSs|SrVrYrrYrYrZrs) isinstancedatetimenp datetime64)srYrYrZ get_converters  z/BaseGroupBy._get_indices..get_converterrNzz,BaseGroupBy._get_indices..zHmust supply a same-length tuple to get_group with multiple grouping keyscsg|]}|qSrYrY)rr)rrYrZrc3s(|]}tddt|DVqdS)css|] \}}||VqdSrVrY)rranrYrYrZ sz5BaseGroupBy._get_indices...N)tuplezipr) convertersrYrZrs&z+BaseGroupBy._get_indices..c3s|]}|VqdSrVrYr) converterrYrZrscsg|] }j|gqSrY)rgetrr]rYrZrs)rrnextiterrr ValueErrorKeyError)rXnames index_sample name_samplemsgerrrY)rrrrXrZ _get_indicess2    zBaseGroupBy._get_indicescCs||gdS)zQ Safe get index, translate keys for datelike to underlying repr. r)r)rXrerYrYrZ _get_indexszBaseGroupBy._get_indexcCs>|jdus t|jtr|jdur|j|jS|jS|j|jSrV) _selectionrrrrGrpr]rYrYrZ _selected_objs    zBaseGroupBy._selected_objset[str]cCs|j|jBSrV)rr_dir_additionsrqr]rYrYrZrszBaseGroupBy._dir_additionsrSa >>> df = pd.DataFrame({'A': 'a b a b'.split(), 'B': [1, 2, 3, 4]}) >>> df A B 0 a 1 1 b 2 2 a 3 3 b 4 To get the difference between each groups maximum and minimum value in one pass, you can do >>> df.groupby('A').pipe(lambda x: x.max() - x.min()) B A a 2 b 2)klassexamplesfunc/Callable[..., T] | tuple[Callable[..., T], str]rcOstj||g|Ri|SrV)compipe)rXrr_r`rYrYrZrszBaseGroupBy.pipeDataFrame | SeriescCs8|dur|j}||}t|st||j||jdS)a Construct DataFrame from group with provided name. Parameters ---------- name : object The name of the group to get as a DataFrame. obj : DataFrame, default None The DataFrame to take the DataFrame out of. If it is None, the object groupby was called on will be used. Returns ------- group : same type as obj Nrs)rrrr_take_with_is_copyrs)rXrerrindsrYrYrZ get_groups  zBaseGroupBy.get_group#Iterator[tuple[Hashable, NDFrameT]]cCsB|j}t|trt|dkrtjdttd|jj |j |j dS)z Groupby iterator. Returns ------- Generator yielding sequence of (name, subsetted object) for each group zIn a future version of pandas, a length 1 tuple will be returned when iterating over a groupby with a grouper equal to a list of length 1. Don't supply a list with a single grouper to avoid this warning. stacklevelr) rtrlistrwarningswarn FutureWarningr&rx get_iteratorrrs)rXrtrYrYrZ__iter__/s  zBaseGroupBy.__iter__)rTr)rTrf)rTr)rTr)rTr)rrrTrrVrTr)rTr)rbrkrlrp__annotations__ frozensetrqr9 _hidden_attrsrtrrrpropertyrrrrrr$rrr#rr"_pipe_templaterrQr\rrrYrYrYrZrnssV     2   rnOutputFrameOrSeries)boundc@s\eZdZUdZded<ded<e          dddd Zd d#d$Zed!d'd(Zed"d)d*Z ed"d+d,Z e d#d.d/Z d$d1d2Z e d%d&d5d6Zed'd9d:Zd(d=d>Ze d)d*dCdDZed+dEdFZ d%d,dIdJZd-dMdNZd.dPdQZedRdSZd/dWdXZeddYdZd[ZeddYd\d]Zeed^jd_ed`dad0dbdcZe  d1d2didjZed dkdldmZe  nd3d4drdsZ d5dwdxZ!e n d6d7dzd{Z" d8d9d|d}Z#eddd~ddZ$ed:ddZ%eddZ&ed;dddZ*ee+ddee,d;d?ddZ-ee+ddee,d;d?ddZ.ee+ddee,d0ddZ/ee+dde+e,de0j1ddfd@ddZ2ee+ddee,e0j1fdAddZ3ee+ddee,ddde0j1fdBddZ4ee+ddee,ddde0j1fdBddZ5ee+ddee,de0j1fdCddZ6ee+ddee,dDddZ7ee8e9dddde0j1dddfdEddZ:ee8e9dddde0j1dfdFddZ;ee8e9dd dnd n  dGdHddZee+dddIdJddZ?ee+ddee,dKddZ@e8eAjBddZBeddZCee+ddee,dLddĄZDee+ddee,dMddDŽZEee+ddee,dNddʄZFed)dOdd΄ZGee+ddd)ddЄZHd)dd҄ZIee+ddd)ddԄZJd)ddքZKee(e+dde+e,ddPddلZL d)dQddބZMedde0j1fdRddZNee+ddd;dSddZOee+ddd;dSddZPee+dde+e,d  dTdUddZQee+ddee,dVd0ddZRee+ddee,dVd0ddZSee+ddee,dWd0ddZTee+ddee,dWd0ddZUee0j1d d d ddfdXddZVee+dddYddZWee+ddee,dZd[ddZXee+ddee,d\ddZYee+dde+e,dd]d^dd ZZee+dde+e,dd]d^d d Z[ed_ddZ\ee]j^dfd`ddZ_e    dadbddZ`dS(crSa Class for grouping and aggregating relational data. See aggregate, transform, and apply functions on this object. It's easiest to use obj.groupby(...) to use GroupBy, but you can also do: :: grouped = groupby(obj, ...) Parameters ---------- obj : pandas object axis : int, default 0 level : int, default None Level of MultiIndex groupings : list of Grouping objects Most users should ignore this exclusions : array-like, optional List of columns to exclude name : str Most users should ignore this Returns ------- **Attributes** groups : dict {group name -> group labels} len(grouped) : int Number of groups Notes ----- After grouping, see aggregate, apply, and transform functions. Here are some other brief notes about usage. When grouping by multiple groups, the result index will be a MultiIndex (hierarchical) by default. Iteration produces (key, group) tuples, i.e. chunking the data by group. So you can write code like: :: grouped = obj.groupby(keys, axis=axis) for key, group in grouped: # do something with the data Function calls on GroupBy, if not specially implemented, "dispatch" to the grouped data. So if you group a DataFrame and wish to invoke the std() method on each group, you can simply do: :: df.groupby(mapper).std() rather than :: df.groupby(mapper).aggregate(np.std) You can pass arguments to these "wrapped" functions, too. See the online documentation for full exposition on these topics and much more rrxboolr{NrTFrrrrtrrsrrvroops.BaseGrouper | Noner}frozenset[Hashable] | None selectionrur~rrzr|ryrwrTrUc Cs||_t|tsJt|||_|s$t|tstd|dkr$td||_||_ | |_ | |_ | |_ | |_ | |_||_|durWddlm}|||||| | |j|jd\}}}||_|||_||_|rlt||_dSt|_dS)Nz(as_index=False only valid with DataFramerz$as_index=False only valid for axis=0 get_grouper)rsrvrur|ryrw)rrr<typervr; TypeErrorrr{rtrur~rzr|ryrwpandas.core.groupby.grouperrrr_get_axis_numberrsrxrr})rXrrrtrsrvrxr}rr{rur~rzr|ryrwrrYrYrZr[s@    zGroupBy.__init__rirfcCsD||jvr t||S||jvr||Stdt|jd|d)N'z' object has no attribute ')_internal_names_setr__getattribute__rrAttributeErrorrrb)rXrirYrYrZrjs   zGroupBy.__getattr__rercsjvsJ'tjttjs+tt fddWdSWdn1s5wYtt jt fdd}|_ |S)Ncs t|SrV)rgr]rerYrZrrz'GroupBy._make_wrapper..csdjvrdddurjd<dtj}fdd}|_tjvr.|Stj v}|r<j j r<j Sj |j || d}j jdkrmjdkrm|jdkrmj j|j}t|dkrmtt|jjrx|rx|}|S)Nrs numeric_onlycsTtd}td|t|gRiWdS1s#wYdS)Nz"The default value of numeric_only ignore)rcatch_warningsfilterwarningsr)xmatch)r_rar`rYrZcurrieds $z7GroupBy._make_wrapper..wrapper..curried) is_transformnot_indexed_samerr) parametersrrsr no_defaultrbr=plotting_methodsrctransformation_kernels_obj_with_exclusionsempty_python_apply_generalrndimcolumns differencer)warn_dropping_nuisance_columns_deprecatedrrxhas_dropped_na_set_result_index_ordered)r_r`rrrresultmissingrarerXsigr^rZwrappers4           z&GroupBy._make_wrapper..wrapper)rq_group_selection_contextrgrrtypes MethodTyperrrcrinspect signaturerb)rXrerrYrrZ _make_wrappers    4zGroupBy._make_wrappercCsz|j}|jr|jdur|jjdkr|jdusdSdd|jD}t|r;|jj}|jt |dd |_| ddSdS)z Create group based selection. Used when selection is not passed directly but instead via a grouper. NOTE: this should be paired with a call to _reset_group_selection NrcSs"g|] }|jdur|jr|jqSrV)rvin_axisre)rgrYrYrZr6s"z0GroupBy._set_group_selection..F)rur) rxr{ groupingsrrrrpr _info_axisrrCtolist _reset_cache)rXgrpgroupersaxrYrYrZ_set_group_selection#s   zGroupBy._set_group_selectioncCs"|jdurd|_|ddSdS)z Clear group based selection. Used for methods needing to return info on each group regardless of whether a group selection was previously set. Nr)rpr r]rYrYrZ_reset_group_selection>s zGroupBy._reset_group_selectionIterator[GroupBy]ccs*|z |VW|dS|w)z; Set / reset the _group_selection_context. N)rrr]rYrYrZrKs z GroupBy._group_selection_contextIterable[Series]cCt|rVr r]rYrYrZ_iterate_slicesVszGroupBy._iterate_slicesroverride_group_keyscsjddlm}fdd}jrA|sA||}jr0jj}jj}jj}||j|||dd} njt t t |} ||j| d} nY|s||jd} j j} jrbjjd} | d k} | | } | jr| jj| st| j}| j|\}}| j|jd} n| j| jdd } n ||}||jd} jjd krjjnj}t| tr|dur|| _| S) Nr)concatcs(tj|D] }|j}|q|SrV)rnot_none _get_axisrs_reset_identity)valuesvr r]rYrZreset_identityes  z/GroupBy._concat_objects..reset_identityF)rsrtlevelsrru)rsrtrrscopyr) pandas.core.reshape.concatrr~r{rx result_indexrrrsrrangerrrrw group_infohas_duplicatesaxesequals algorithmsunique1d_valuesindexget_indexer_non_uniquetakereindexrrrrerrrG)rXrrrrrr~ group_levels group_namesrrtr labelsmasktargetindexer_rerYr]rZ_concat_objects\sH      zGroupBy._concat_objectsrrcCs|j|j}|jjr|jjs|j||jdd}|St|j}|j||jdd}|j |jd}|jjrA|j t t ||jd}|j||jdd}|S)NFrr) rrrrsrx is_monotonicrset_axisrC result_ilocs sort_indexr.rEr)rXrobj_axisoriginal_positionsrYrYrZrsz!GroupBy._set_result_index_ordered"Mapping[base.OutputKey, ArrayLike]Series | DataFramecCrrVrrXrrYrYrZ_indexed_output_to_ndframerz"GroupBy._indexed_output_to_ndframeoutput7Series | DataFrame | Mapping[base.OutputKey, ArrayLike]qsnpt.NDArray[np.float64] | NonecCst|ttfr |}n||}|js$|||}tt|j j }n|j j }|dur1t ||}||_ |jdkrK|j}|j |jj rK|jj |_ |j||dS)a Wraps the output of GroupBy aggregations into the expected result. Parameters ---------- output : Series, DataFrame, or Mapping[base.OutputKey, ArrayLike] Data to wrap. Returns ------- Series or DataFrame NrrC)rrGr;r@r{_insert_inaxis_grouper_inplace _consolidaterCr#rxrr"_insert_quantile_levelr+rsrr'rrr _reindex_output)rXrArCrr+rYrYrZ_wrap_aggregated_outputs     zGroupBy._wrap_aggregated_outputcCsFt|ttfr |}n||}|jdkr|j}|jj|_|jj|_|S)aN Wraps the output of GroupBy transformations into the expected result. Parameters ---------- output : Mapping[base.OutputKey, ArrayLike] Data to wrap. Returns ------- Series or DataFrame Series for SeriesGroupBy, DataFrame for DataFrameGroupBy r) rrGr;r@rsrrrrr+)rXrArrYrYrZ_wrap_transformed_outputs    z GroupBy._wrap_transformed_outputrrcCrrVr)rXdatarrrrYrYrZ_wrap_applied_outputszGroupBy._wrap_applied_outputhowrc Cs|tjur0|jjdkr.|dk}|jr|jj}n|j}|}t|j r-t|j s-|j s-d}nd}|re|jjdkret |jj set jt|jd|d|d|jj dttdtt|jd|d |S) a Determine subclass-specific default value for 'numeric_only'. For SeriesGroupBy we want the default to be False (to match Series behavior). For DataFrameGroupBy we want it to be True (for backwards-compat). Parameters ---------- numeric_only : bool or lib.no_default axis : int Axis passed to the groupby op (not self.axis). Returns ------- bool rF.z called with numeric_only= and dtype z;. This will raise a TypeError in a future version of pandas)categoryrz does not implement numeric_only)rrrrrrsrr_get_numeric_datarrrr-dtyperrrrbrr&NotImplementedError)rXrNrrsrrcheckrYrYrZ_resolve_numeric_only#s2   zGroupBy._resolve_numeric_onlyrcCsL|jjdkr |jdkr"t|jt|jjkr$tt|||dSdSdSdS)a:Emit warning on numeric_only behavior deprecation when appropriate. Parameters ---------- how : str Groupby kernel name. result : Result of the groupby operation. numeric_only : bool or lib.no_default Argument as passed by user. rN)rrrrrr)rXrNrrrYrYrZ_maybe_warn_numeric_only_deprXs  z%GroupBy._maybe_warn_numeric_only_deprc Cs|jj\}}}t||}tj||dd}|j||jd}t|jj dkr*t d|j }t |t r>|jj dj} || }||} t||\} } | | | |fS)NF) allow_fillrrzAMore than 1 grouping labels are not supported with engine='numba'r)rxr$rHr(take_ndr-rsto_numpyrrrUr+rrDreget_level_valuesrgenerate_slices) rXrLidsr5r sorted_index sorted_ids sorted_data index_data group_keysorted_index_datastartsendsrYrYrZ _numba_prepps&   zGroupBy._numba_prepr engine_kwargsdict[str, bool] | NonecGs|jstd|jdkrtd| |j}Wdn1s"wY|jdkr.|n|}||\}}}} tj |fit |} | | ||dg|R} |j j } |jdkrdd|j i} | } nd|ji} |j| fd | i| S) zp Perform groupby with a standard numerical aggregation function (e.g. mean) with Numba. z validate_udfgenerate_numba_transform_funcrIrrr-rargsort) rXrLrrhr_r`rerfr_ranumba_transform_funcrrYrYrZ_transform_with_numbas"  zGroupBy._transform_with_numbac OsV||\}}}} t|tj|fit||} | | |||t|jg|R} | S)a* Perform groupby aggregation routine with the numba engine. This routine mimics the data splitting routine of the DataSplitter class to generate the indices of each group in the sorted data and then passes the data and indices into a Numba jitted function. )rgr>rsgenerate_numba_agg_funcrIrr) rXrLrrhr_r`rerfr_ranumba_agg_funcrrYrYrZ_aggregate_with_numbas" zGroupBy._aggregate_with_numbarN dataframerO)inputrc sjtttr4t|r,t|}t|r|iSs#r*td|Stdds8r\trIt fdd}ntt drXtt d}ntd}t ddHz | ||j }Wn-ty|| ||j WdYWdS1swYYn wWd|SWd|S1swY|S) Nz"Cannot pass arguments to property z$apply func should be callable, not 'rcsFtjdd|gRiWdS1swYdS)Nr)all)rerrstate)rr_rr`rYrZras$zGroupBy.apply..fnanz6func must be a callable if args or kwargs are suppliedzmode.chained_assignment)ris_builtin_funcrrfhasattrrgcallablerrrr3rrrr)rXrr_r`resrarrYrrZrcsL          z GroupBy.applyrarL bool | Noneris_aggc Cs|j|||j\}}|dur|p|j}d}|ot|dk} |s7|jtjur7|s7| s7d} tj | t t dd}|j ||||p?|dS)aK Apply function f in python space Parameters ---------- f : callable Function to apply data : Series or DataFrame Data to apply f to not_indexed_same: bool, optional When specified, overrides the value of not_indexed_same. Apply behaves differently when the result index is equal to the input index, but this can be coincidental leading to value-dependent behavior. is_transform : bool, default False Indicator for whether the function is actually a transform and should not have group keys prepended. This is used in _make_wrapper which generates both transforms (e.g. diff) and non-transforms (e.g. corr) is_agg : bool, default False Indicator for whether the function is an aggregation. When the result is empty, we don't want to warn for this case. See _GroupBy._python_agg_general. Returns ------- Series or DataFrame data after applying f NFra|Not prepending group keys to the result index of transform-like apply. In the future, the group keys will be included in the index, regardless of whether the applied function returns a like-indexed object. To preserve the previous behavior, use >>> .groupby(..., group_keys=False) To adopt the future behavior and silence this warning, use >>> .groupby(..., group_keys=True)rT)r) rxrcrsryrr~rrrrrr&rM) rXrarLrrrrryr is_empty_aggrrYrYrZrs(%  zGroupBy._python_apply_general)raise_on_typeerrorc stfdd}i}|jdkr|j||jddSt|D]1\}}|j} z |j ||} Wnt yH|r=t t |dddYq#wt j| |d } | || <q#|s^|||jS||S) Ncs|gRiSrVrYrrrYrZrhrz-GroupBy._python_agg_general..rT)raggFr)labelposition)rrrrr enumeraterrerx agg_seriesrrrr= OutputKeyrJ) rXrrr_r`rarAidxrrrerrrYrrZ_python_agg_generales,      zGroupBy._python_agg_generalr min_countaliasnpfunccCsN||j||||d}|j|jddWdS1s wYdS)N)rNaltrrrRmethod)r_cython_agg_general __finalize__rr)rXrrrrrrYrYrZ _agg_generals $zGroupBy._agg_generalrrrcCs~|jdkr t|}nt|j}|jddksJ|jdddf}|jj||dd}t|t r9t |j ||j d}t ||dS)zn Fallback to pure-python aggregation if _cython_operation raises NotImplementedError. rNrT)preserve_dtyperT)r)rrGr;rshapeilocrxrrr7r_from_sequencerTrF)rXrrrserro res_valuesrYrYrZ_agg_py_fallbacks    zGroupBy._agg_py_fallbackignore_failuresc sj|dd}jdk}t} |r?|r7tjjs7d} dvr'd} ttj dd| d|s?j d d dfdd } j | |d} |sbt| | krbt t| | } |rsjj| _| S| S)Nrrrranyr} bool_onlyrPz does not implement Fr rrrTcsRzjjd|fjdd}W|Sty(j|jd}Y|Sw)N aggregaterrsr)rr)rx_cython_operationrrUr)rrrrLrNr`rrXrYrZ array_funcs  z/GroupBy._cython_agg_general..array_funcrrrrTr)rW_get_data_to_aggregaterrr-rrTrUrrbget_numeric_datagrouped_reducer_wrap_agged_managerrxr"r+rI)rXrNrrrrr`numeric_only_boolis_serorig_lenkwd_namernew_mgrrrYrrZrs.      zGroupBy._cython_agg_generalcKrrVr)rXrNrrsr`rYrYrZ_cython_transformrzGroupBy._cython_transform)enginerhc Ostt|rY| |j}Wdn1swY|jdkr"|n|}|j||g|Rd|i|}|jjdkrItt|jj ||j |j dStt |jj | |j |jdSt|p_|}t|tsq|j|g|Ri|S|tjvrd|d} t| |tjvs|tjvrt|||i|St|ddt|||i|}Wdn1swY||S) NrOrhr+rr+rerz2' is not a valid function name for transform(name)r|T)rJrrrrjrwrrrr;rmr+rrGrlrerget_cython_funcrrf_transform_generalr=transform_kernel_allowlistrcythonized_kernelsrrg temp_setattr_wrap_transform_fast_result) rXrrrhr_r`rLrorrrYrYrZ _transformsB          zGroupBy._transformcCs|j}|jj\}}}|j|jj|jdd}|jjdkr.t |j |}|j ||j |j d}|S|jdkr5dn|j}|j||dd}|j||j|d}|S)z7 Fast transform path for aggregations. Frrrr)rsconvert_indicesr)rrxr$r.r"rsrrrr(rZr*rmr+re_taker8r)rXrrrr^r5outrArsrYrYrZr5s  z#GroupBy._wrap_transform_fast_resultcCst|dkrtjgdd}ntt|}|r#|jj||jd}|Stjt|jj t d}| dd|| t <t|t|jjdddgj}|j|}|S)Nrint64rrFTr)rrarrayru concatenaterr-rsrr+rfillastypertilerrrwhere)rXrrwfilteredr2rYrYrZ _apply_filterQs  $ zGroupBy._apply_filter ascending np.ndarrayc Cs2|jj\}}}t||}||t|}}|dkr!tjdtjdStjd|dd|ddkf}ttjt |d|f}| } |rS| t | ||8} nt | tj|dddf|| } |jj r{t |dktj| jtjdd} n| jtjdd} tj|tjd} tj|tjd| |<| | S) a. Parameters ---------- ascending : bool, default True If False, number in reverse, from length of group - 1 to 0. Notes ----- this is currently implementing sort=False (though the default is sort=True) for groupby in general rrTNrrFr)rxr$rHrrrrr_diffnonzerocumsumrepeatrrrrfloat64intparange) rXrr^r5rsortercountrunreprrevrYrYrZ_cumcount_arraybs" " &"zGroupBy._cumcount_arraycCs,t|jtr |jjSt|jtsJ|jjSrV)rrrr;_constructor_slicedrGrmr]rYrYrZ_obj_1d_constructors zGroupBy._obj_1d_constructorval_testLiteral['any', 'all']skipnac sBdfdd } dddd}|jtjdttjdd|||d S)zO Shared func to call any / all Cython GroupBy implementations. valsrrTtuple[np.ndarray, type]cst|jr$rtjddtgd}||}n|jtdd}ttj|}nt|t r2|j jtdd}n|jtdd}| tj tfS)NcSst|st|SdS)NT)r1rrrYrYrZrrz9GroupBy._bool_agg..objs_to_bool..)otypesFr) r.rTr vectorizerrrndarrayrr5_dataviewint8)rrrrYrZ objs_to_bools    z'GroupBy._bool_agg..objs_to_boolFrr inferencernullablercSs*|rt|jtdd|dkS|j|ddS)NFrr)r6rr)rrrrYrYrZresult_to_boolsz)GroupBy._bool_agg..result_to_boolT)r cython_dtype needs_maskneeds_nullablepre_processingpost_processingrrN)rrrTr)F)rrrrrrrTr)_get_cythonized_result libgroupby group_any_allrrTr)rXrrrrrYrrZ _bool_aggs   zGroupBy._bool_aggrRrcC |d|S)a Return True if any value in the group is truthful, else False. Parameters ---------- skipna : bool, default True Flag to ignore nan values during truth testing. Returns ------- Series or DataFrame DataFrame or Series of boolean values, where a value is True if any element is True within its respective group, False otherwise. rrrXrrYrYrZr z GroupBy.anycCr)a Return True if all values in the group are truthful, else False. Parameters ---------- skipna : bool, default True Flag to ignore nan values during truth testing. Returns ------- Series or DataFrame DataFrame or Series of boolean values, where a value is True if all elements are True within its respective group, False otherwise. r}rrrYrYrZr}rz GroupBy.allcs|}|jj\}dk|jdkd fdd }||}t|dd  ||}Wd n1s:wY|jdkrI|jj|_ |j |d d S)z Compute count of group, excluding missing values. Returns ------- Series or DataFrame Count of values within each group. rrbvaluesrrTcsr|jdkrt|dd@}nt|@}tj|dd}r7|jdks*J|jddks3J|dS|S)Nrr)r1max_binrsrOr)rr1reshapercount_level_2dr)rmaskedcountedr^ is_seriesr2rrYrZhfuncs zGroupBy.count..hfuncr|TNr fill_value)rrrTr) rrxr$rrrrrr"r+rI)rXrLr5rrrrYrrZrs      z GroupBy.count)see_alsocythonrcsX|jd|ddt|rddlm}|||S|jdfdd|d}|j|jdd S) a, Compute mean of groups, excluding missing values. Parameters ---------- numeric_only : bool, default True Include only float, int, boolean columns. If None, will attempt to use everything, then use only numeric data. engine : str, default None * ``'cython'`` : Runs the operation through C-extensions from cython. * ``'numba'`` : Runs the operation through JIT compiled code from numba. * ``None`` : Defaults to ``'cython'`` or globally setting ``compute.use_numba`` .. versionadded:: 1.4.0 engine_kwargs : dict, default None * For ``'cython'`` engine, there are no accepted ``engine_kwargs`` * For ``'numba'`` engine, the engine can accept ``nopython``, ``nogil`` and ``parallel`` dictionary keys. The values must either be ``True`` or ``False``. The default ``engine_kwargs`` for the ``'numba'`` engine is ``{{'nopython': True, 'nogil': False, 'parallel': False}}`` .. versionadded:: 1.4.0 Returns ------- pandas.Series or pandas.DataFrame %(see_also)s Examples -------- >>> df = pd.DataFrame({'A': [1, 1, 2, 1, 2], ... 'B': [np.nan, 2, 3, 4, 5], ... 'C': [1, 2, 1, 1, 2]}, columns=['A', 'B', 'C']) Groupby one column and return the mean of the remaining columns in each group. >>> df.groupby('A').mean() B C A 1 3.0 1.333333 2 4.0 1.500000 Groupby two columns and return the mean of the remaining column. >>> df.groupby(['A', 'B']).mean() C A B 1 2.0 2.0 4.0 1.0 2 3.0 1.0 5.0 2.0 Groupby one column and return the mean of only particular column in the group. >>> df.groupby('A')['B'].mean() A 1 3.0 2 4.0 Name: B, dtype: float64 meanrr) sliding_meanct|jdSNr)rGr rrrYrZrjzGroupBy.mean..rrrRr)rWrJpandas.core._numba.kernelsr rrrrrr)rXrrrhr rrYrrZr sI   z GroupBy.meancs8|jd|dd|jdfdd|d}|j|jddS) a Compute median of groups, excluding missing values. For multiple groupings, the result index will be a MultiIndex Parameters ---------- numeric_only : bool, default True Include only float, int, boolean columns. If None, will attempt to use everything, then use only numeric data. Returns ------- Series or DataFrame Median of values within each group. medianrrcrr)rGrrrrYrZrrz GroupBy.median..rrRr)rWrrrr)rXrrrYrrZros zGroupBy.medianrddof str | NonecCst|rddlm}t||||S|jd|dd}|r;|jjdkr;t |jj s;t t |j d|d|jj |jtjt tj|dd d |d }|d|||S) aE Compute standard deviation of groups, excluding missing values. For multiple groupings, the result index will be a MultiIndex. Parameters ---------- ddof : int, default 1 Degrees of freedom. engine : str, default None * ``'cython'`` : Runs the operation through C-extensions from cython. * ``'numba'`` : Runs the operation through JIT compiled code from numba. * ``None`` : Defaults to ``'cython'`` or globally setting ``compute.use_numba`` .. versionadded:: 1.4.0 engine_kwargs : dict, default None * For ``'cython'`` engine, there are no accepted ``engine_kwargs`` * For ``'numba'`` engine, the engine can accept ``nopython``, ``nogil`` and ``parallel`` dictionary keys. The values must either be ``True`` or ``False``. The default ``engine_kwargs`` for the ``'numba'`` engine is ``{{'nopython': True, 'nogil': False, 'parallel': False}}`` .. versionadded:: 1.4.0 numeric_only : bool, default True Include only `float`, `int` or `boolean` data. .. versionadded:: 1.5.0 Returns ------- Series or DataFrame Standard deviation of values within each group. r sliding_varstdrrz.std called with numeric_only=rQTcSrrV)rsqrtrrrYrYrZrrzGroupBy.std..)rr needs_countsrr)rJrrrrrrrWrrrr-rTrrrbrr group_varrrX)rXrrrhrrrrrYrYrZrs6/     z GroupBy.stdcsDt|rddlm}|||S|jdfdd||tjudS)a1 Compute variance of groups, excluding missing values. For multiple groupings, the result index will be a MultiIndex. Parameters ---------- ddof : int, default 1 Degrees of freedom. engine : str, default None * ``'cython'`` : Runs the operation through C-extensions from cython. * ``'numba'`` : Runs the operation through JIT compiled code from numba. * ``None`` : Defaults to ``'cython'`` or globally setting ``compute.use_numba`` .. versionadded:: 1.4.0 engine_kwargs : dict, default None * For ``'cython'`` engine, there are no accepted ``engine_kwargs`` * For ``'numba'`` engine, the engine can accept ``nopython``, ``nogil`` and ``parallel`` dictionary keys. The values must either be ``True`` or ``False``. The default ``engine_kwargs`` for the ``'numba'`` engine is ``{{'nopython': True, 'nogil': False, 'parallel': False}}`` .. versionadded:: 1.4.0 numeric_only : bool, default True Include only `float`, `int` or `boolean` data. .. versionadded:: 1.5.0 Returns ------- Series or DataFrame Variance of values within each group. rrvarcr)Nr)rGrrrrYrZr rzGroupBy.var..)rrrr)rJrrrrrrr)rXrrrhrrrYrrZrs/  z GroupBy.varc Cs$|jd|dd}|r'|jjdkr't|jjs'tt|jd|d|jj|j||d}| d|||jdkrE|t | }|S|j |j}| }|j |}|j |}t&tdd |jd d |ft |jd d |f<Wd |S1swY|S) a Compute standard error of the mean of groups, excluding missing values. For multiple groupings, the result index will be a MultiIndex. Parameters ---------- ddof : int, default 1 Degrees of freedom. numeric_only : bool, default True Include only `float`, `int` or `boolean` data. .. versionadded:: 1.5.0 Returns ------- Series or DataFrame Standard error of the mean of values within each group. semrrrz.sem called with numeric_only=rQ)rrrz*.*will attempt to set the values inplace.*N)rWrrrr-rTrrrbrrXrrrrrr}uniqueget_indexer_forrrrr) rXrrrrcolscountsr9 count_ilocsrYrYrZr  s@        0 z GroupBy.semcCsV|j}t|jtr|j||jjd}n||}|js$|d }|j |ddS)z Compute group sizes. Returns ------- DataFrame or Series Number of rows in each group as a Series if as_index is True or a DataFrame if as_index is False. rsizerr) rxr&rrrrGrrer{rename reset_indexrIr?rYrYrZr&H s  z GroupBy.sizesum)fnamenomccCspt|rddlm}|||St|dd|j||dtjd}Wdn1s,wY|j |ddS)Nr) sliding_sumr|Tr)rrrrr) rJrr-rrrrrrr)rI)rXrrrrhr-rrYrYrZr)d s z GroupBy.sumprodcCs|j||dtjdS)Nr/r.)rrr/)rXrrrYrYrZr/ s z GroupBy.prodmincC6t|rddlm}|||dS|j||dtjdS)Nrsliding_min_maxFr0r.)rJrr3rrrrr0rXrrrrhr3rYrYrZr0  z GroupBy.minmaxcCr1)Nrr2Tr6r.)rJrr3rrrrr6r4rYrYrZr6 r5z GroupBy.maxcCd d dd}|j||d|d S) a Compute the first non-null entry of each column. Parameters ---------- numeric_only : bool, default False Include only float, int, boolean columns. min_count : int, default -1 The required number of valid values to perform the operation. If fewer than ``min_count`` non-NA values are present the result will be NA. Returns ------- Series or DataFrame First non-null of values within each group. See Also -------- DataFrame.groupby : Apply a function groupby to each row or column of a DataFrame. DataFrame.core.groupby.GroupBy.last : Compute the last non-null entry of each column. DataFrame.core.groupby.GroupBy.nth : Take the nth row from each group. Examples -------- >>> df = pd.DataFrame(dict(A=[1, 1, 3], B=[None, 5, 6], C=[1, 2, 3], ... D=['3/11/2000', '3/12/2000', '3/13/2000'])) >>> df['D'] = pd.to_datetime(df['D']) >>> df.groupby("A").first() B C D A 1 5.0 1 2000-03-11 3 6.0 3 2000-03-13 >>> df.groupby("A").first(min_count=2) B C D A 1 NaN 1.0 2000-03-11 3 NaN NaN NaT >>> df.groupby("A").first(numeric_only=True) B C A 1 5.0 1 3 6.0 3 rrrrrsrcS@ddd}t|tr|j||dSt|tr||Stt|)NrrGcS&|jt|j}t|stjS|dS)z-Helper function for first item that isn't NA.rrr2rrrrarrrYrYrZfirst z2GroupBy.first..first_compat..firstrrrGrr;rcrGrr)rrrsr=rYrYrZ first_compat    z#GroupBy.first..first_compatr=r.Nrrrrrsrr)rXrrrArYrYrZr= s 1z GroupBy.firstcCr7) an Compute the last non-null entry of each column. Parameters ---------- numeric_only : bool, default False Include only float, int, boolean columns. If None, will attempt to use everything, then use only numeric data. min_count : int, default -1 The required number of valid values to perform the operation. If fewer than ``min_count`` non-NA values are present the result will be NA. Returns ------- Series or DataFrame Last non-null of values within each group. See Also -------- DataFrame.groupby : Apply a function groupby to each row or column of a DataFrame. DataFrame.core.groupby.GroupBy.first : Compute the first non-null entry of each column. DataFrame.core.groupby.GroupBy.nth : Take the nth row from each group. Examples -------- >>> df = pd.DataFrame(dict(A=[1, 1, 3], B=[5, None, 6], C=[1, 2, 3])) >>> df.groupby("A").last() B C A 1 5.0 2 3 6.0 3 rrrrrsrcSr8)NrrGcSr9)z,Helper function for last item that isn't NA.rr:r;rYrYrZlast# r>z/GroupBy.last..last_compat..lastrr?r@)rrrsrFrYrYrZ last_compat" rBz!GroupBy.last..last_compatrFr.NrCrDrE)rXrrrGrYrYrZrF s &z GroupBy.lastr;cCsz|jjdkr4|j}t|j}|std|jjd|jdddd}gd}|jj ||jj |d }| |S| d d |j S) a Compute open, high, low and close values of a group, excluding missing values. For multiple groupings, the result index will be a MultiIndex Returns ------- DataFrame Open, high, low and close values within each group. rzNo numeric types to aggregaterohlcrrr)openhighlowclosercSs|SrV)rHrrYrYrZrY rzGroupBy.ohlc..)rrrrr-rTr!rxrr*_constructor_expanddimr"rI_apply_to_column_groupbysr)rXrr is_numericr agg_namesrrYrYrZrH8 s      z GroupBy.ohlcc s|Vt|jdkr3|jjdi}|jjdkr|}n|}|jjddWdS|j fdd|jdd}|j dkrO|jWdS|WdS1s]wYdS)Nrrcs|jdiS)NrY)describerr`rYrZrh rz"GroupBy.describe..T)rrY) rrrrQrunstackrjrrrrs)rXr` describedrrYrRrZrQ\ s$    $zGroupBy.describecOs$ddlm}|||g|Ri|S)a< Provide resampling when using a TimeGrouper. Given a grouper, the function resamples it according to a string "string" -> "frequency". See the :ref:`frequency aliases ` documentation for more details. Parameters ---------- rule : str or DateOffset The offset string or object representing target grouper conversion. *args, **kwargs Possible arguments are `how`, `fill_method`, `limit`, `kind` and `on`, and other arguments of `TimeGrouper`. Returns ------- Grouper Return a new grouper with our resampler appended. See Also -------- Grouper : Specify a frequency to resample with when grouping by a key. DatetimeIndex.resample : Frequency conversion and resampling of time series. Examples -------- >>> idx = pd.date_range('1/1/2000', periods=4, freq='T') >>> df = pd.DataFrame(data=4 * [range(2)], ... index=idx, ... columns=['a', 'b']) >>> df.iloc[2, 0] = 5 >>> df a b 2000-01-01 00:00:00 0 1 2000-01-01 00:01:00 0 1 2000-01-01 00:02:00 5 1 2000-01-01 00:03:00 0 1 Downsample the DataFrame into 3 minute bins and sum the values of the timestamps falling into a bin. >>> df.groupby('a').resample('3T').sum() a b a 0 2000-01-01 00:00:00 0 2 2000-01-01 00:03:00 0 1 5 2000-01-01 00:00:00 5 1 Upsample the series into 30 second bins. >>> df.groupby('a').resample('30S').sum() a b a 0 2000-01-01 00:00:00 0 1 2000-01-01 00:00:30 0 0 2000-01-01 00:01:00 0 1 2000-01-01 00:01:30 0 0 2000-01-01 00:02:00 0 0 2000-01-01 00:02:30 0 0 2000-01-01 00:03:00 0 1 5 2000-01-01 00:02:00 5 1 Resample by month. Values are assigned to the month of the period. >>> df.groupby('a').resample('M').sum() a b a 0 2000-01-31 0 3 5 2000-01-31 5 1 Downsample the series into 3 minute bins as above, but close the right side of the bin interval. >>> df.groupby('a').resample('3T', closed='right').sum() a b a 0 1999-12-31 23:57:00 0 1 2000-01-01 00:00:00 0 2 5 2000-01-01 00:00:00 5 1 Downsample the series into 3 minute bins and close the right side of the bin interval, but label each bin using the right edge instead of the left. >>> df.groupby('a').resample('3T', closed='right', label='right').sum() a b a 0 2000-01-01 00:00:00 0 1 2000-01-01 00:03:00 0 2 5 2000-01-01 00:03:00 5 1 r)get_resampler_for_grouping)pandas.core.resamplerU)rXruler_r`rUrYrYrZresamplep s bzGroupBy.resamplerMcOs.ddlm}||jg|R|j|jd|S)zV Return a rolling grouper, providing rolling functionality per group. r)rM)_grouper _as_index)pandas.core.windowrMrrxr{)rXr_r`rMrYrYrZrolling s zGroupBy.rollingrKcO*ddlm}||jg|Rd|ji|S)zc Return an expanding grouper, providing expanding functionality per group. r)rKrY)r[rKrrx)rXr_r`rKrYrYrZ expanding s zGroupBy.expandingrLcOr])zO Return an ewm grouper, providing ewm functionality per group. r)rLrY)r[rLrrx)rXr_r`rLrYrYrZewm s z GroupBy.ewm directionLiteral['ffill', 'bfill']c s|durd}jj\}}}tj|ddjtjdd}|dkr%|ddd}ttj||||j ddfd d }j }j dkrD|j }|j }||} || } t| trZ|j| _| S)a  Shared function for `pad` and `backfill` to call Cython method. Parameters ---------- direction : {'ffill', 'bfill'} Direction passed to underlying Cython function. `bfill` will cause values to be filled backwards. `ffill` and any other values will default to a forward fill limit : int, default None Maximum number of consecutive values to fill. If `None`, this method will convert to -1 prior to passing to Cython Returns ------- `Series` or `DataFrame` with filled values See Also -------- pad : Returns Series with minimum number of char in object. backfill : Backward fill the missing values in the dataset. Nr mergesort)kindFrbfill)r1 sorted_labelsr`limitrwrrrTcst|}|jdkrtj|jtjd}||dt||St|tj r9|j }j j r0t |j }tj|j|d}n t|j|j|j d}tt|D]#}tj|jdtjd}|||dt|||||ddf<qJ|S)Nrr)rr2)r1rrrrrr(rZrrrTrxrr'r_emptyr#r)rr2r4rTricol_funcrXrYrZblk_func2 s      zGroupBy._fill..blk_funcrr)rxr$rrurrrrgroup_fillna_indexerrwrrsr_mgrrcrmrrGrerK) rXr`rfr^r5rerkrrmgrres_mgrnew_objrYrirZ_fill s0      z GroupBy._fillcC|jd|dS)a: Forward fill the values. Parameters ---------- limit : int, optional Limit of how many values to fill. Returns ------- Series or DataFrame Object with missing values filled. See Also -------- Series.ffill: Returns Series with minimum number of char in object. DataFrame.ffill: Object with missing values filled or None if inplace=True. Series.fillna: Fill NaN values of a Series. DataFrame.fillna: Fill NaN values of a DataFrame. ffillrfrqrXrfrYrYrZrs] z GroupBy.ffillcCtjdttd|j|dS)aE Forward fill the values. .. deprecated:: 1.4 Use ffill instead. Parameters ---------- limit : int, optional Limit of how many values to fill. Returns ------- Series or DataFrame Object with missing values filled. zMpad is deprecated and will be removed in a future version. Use ffill instead.rrt)rrrr&rsrvrYrYrZpadv  z GroupBy.padcCrr)a/ Backward fill the values. Parameters ---------- limit : int, optional Limit of how many values to fill. Returns ------- Series or DataFrame Object with missing values filled. See Also -------- Series.bfill : Backward fill the missing values in the dataset. DataFrame.bfill: Backward fill the missing values in the dataset. Series.fillna: Fill NaN values of a Series. DataFrame.fillna: Fill NaN values of a DataFrame. rdrtrurvrYrYrZrd rwz GroupBy.bfillcCrx)aF Backward fill the values. .. deprecated:: 1.4 Use bfill instead. Parameters ---------- limit : int, optional Limit of how many values to fill. Returns ------- Series or DataFrame Object with missing values filled. zRbackfill is deprecated and will be removed in a future version. Use bfill instead.rrt)rrrr&rdrvrYrYrZbackfill rzzGroupBy.backfillrAcCr)ae Take the nth row from each group if n is an int, otherwise a subset of rows. Can be either a call or an index. dropna is not available with index notation. Index notation accepts a comma separated list of integers and slices. If dropna, will take the nth non-null row, dropna is either 'all' or 'any'; this is equivalent to calling dropna(how=dropna) before the groupby. Parameters ---------- n : int, slice or list of ints and slices A single nth value for the row or a list of nth values or slices. .. versionchanged:: 1.4.0 Added slice and lists containing slices. Added index notation. dropna : {'any', 'all', None}, default None Apply the specified dropna operation before counting which row is the nth row. Only supported if n is an int. Returns ------- Series or DataFrame N-th value within each group. %(see_also)s Examples -------- >>> df = pd.DataFrame({'A': [1, 1, 2, 1, 2], ... 'B': [np.nan, 2, 3, 4, 5]}, columns=['A', 'B']) >>> g = df.groupby('A') >>> g.nth(0) B A 1 NaN 2 3.0 >>> g.nth(1) B A 1 2.0 2 5.0 >>> g.nth(-1) B A 1 4.0 2 5.0 >>> g.nth([0, 1]) B A 1 NaN 1 2.0 2 3.0 2 5.0 >>> g.nth(slice(None, -1)) B A 1 NaN 1 2.0 2 3.0 Index notation may also be used >>> g.nth[0, 1] B A 1 NaN 1 2.0 2 3.0 2 5.0 >>> g.nth[:-1] B A 1 NaN 1 2.0 2 3.0 Specifying `dropna` allows count ignoring ``NaN`` >>> g.nth(0, dropna='any') B A 1 2.0 2 3.0 NaNs denote group exhausted when using dropna >>> g.nth(3, dropna='any') B A 1 NaN 2 NaN Specifying `as_index=False` in `groupby` keeps the original index. >>> df.groupby('A', as_index=False).nth(1) A B 1 1 2.0 4 2 5.0 )rAr]rYrYrZnth skz GroupBy.nthrPositionalIndexer | tupleLiteral['any', 'all', None]cCs0|so|a||}|jj\}}}||dk@}||}|js*|WdS|jj}|jdkrM||||_|j sGt |t rG| |}| |}n||||_|jr^|j|jdn|WdS1sjwYt|swtd|dvrtd|dtt|}|dkr|nd|}|jj||jd} |jdur|jdur|jj} | | | j} ndd lm} | | |j|j|j|j|jd \} }}| j| |j|j|jd } | | |}}||kj }t!|r|"rt#j$|j%|<t!|jt!| kst!|t!|jjkr|jj|_|S| |jj}|S) Nrrrz4dropna option only supported for an integer argumentrz_For a DataFrame or Series groupby.nth, dropna must be either None, 'any' or 'all', (was passed z).)rNrsr)rrsrvrury)r{rurs)&r"_make_mask_from_positional_indexerrxr$_mask_selected_objr{r"rsr+r|rrBr.rIrrur:r+rrrrrrwrtrvisinrrryrRr&r|r*rrrrloc)rXrrwr2r^r5rr"max_lendroppedrsrxrgrbsizesrrYrYrZ_nth0 sn               z GroupBy._nthg?linear interpolationcs|jd|dd}|r'|jjdkr't|jjs'tt|jd|d|jjdd d dfdd t|}|r<|g}t j |t j d}|j j \}}t|ttj||dt|dkrd|dnd} t |dk| |dfdd } |j} | jdk} |} |r| n| }|}|j| |d}|tjur| st|jt| jkrtt|d|t|jdkr| j| ddtd| r||}n| |}|r||S|j||dS) a Return group values at the given quantile, a la numpy.percentile. Parameters ---------- q : float or array-like, default 0.5 (50% quantile) Value(s) between 0 and 1 providing the quantile(s) to compute. interpolation : {'linear', 'lower', 'higher', 'midpoint', 'nearest'} Method to use when the desired quantile falls between two points. numeric_only : bool, default True Include only `float`, `int` or `boolean` data. .. versionadded:: 1.5.0 Returns ------- Series or DataFrame Return type determined by caller of GroupBy object. See Also -------- Series.quantile : Similar method for Series. DataFrame.quantile : Similar method for DataFrame. numpy.percentile : NumPy method to compute qth percentile. Examples -------- >>> df = pd.DataFrame([ ... ['a', 1], ['a', 2], ['a', 3], ... ['b', 1], ['b', 3], ['b', 5] ... ], columns=['key', 'val']) >>> df.groupby('key').quantile() val key a 2.0 b 3.0 quantilerrrz#.quantile called with numeric_only=rQrrrT"tuple[np.ndarray, np.dtype | None]cSst|rtdd}t|jr)t|tr|jttj d}n|}ttj }||fSt |jr?t|tr?|jttj d}||fSt |jrUtd}t |t}||fSt|jrktd}t |t}||fSt|trt|rttj}|jttj d}||fSt |}||fS)Nz7'quantile' cannot be performed against 'object' dtypes!)rTna_valuezdatetime64[ns]ztimedelta64[ns])r.rr,rTrr8r[floatrrrr(r)asarrayrr0r*r)rrrrYrYrZ pre_processor s8       z'GroupBy.quantile..pre_processorrrnp.dtype | Nonecs"|rt|r dvs||}|S)N>rmidpoint)r,rr)rrYrZpost_processor s z(GroupBy.quantile..post_processorr)r1rCrrrc st|}|\}}d}|jdkr!|jd}t|tf}n}tj|ftjd}||f}t|j tj dd}|jdkrM|d|||dnt |D]} || || || || dqQ|jdkro| d}n| |}||S) NrrOrrFr)rr2 sort_indexerK)r1rrr broadcast_torrrlexsortrrr#rlr) rr2rrncols shaped_labelsrordersort_arrrh)rlabels_for_lexsortrnqsrrrYrZrk s(      "   z"GroupBy.quantile..blk_funcrF*All columns were dropped in grouped_reducerEN)rrrTr)rrrrrTrr)rWrrrr-rTrrrbr/rrrrxr$rrrgroup_quantiler6rrrrrrritemsrrrmrJ)rXqrrr orig_scalarrCr^r5na_label_for_sortingrkrrrrnrLrrorrY)rrrrrrrrZr sl,           zGroupBy.quantilecCs|:|jj}|jjd}|jjr!t|dktj|}tj }ntj }|j |||d}|s5|j d|}|WdS1sAwYdS)a) Number each group from 0 to the number of groups - 1. This is the enumerative complement of cumcount. Note that the numbers given to the groups match the order in which the groups would be seen when iterating over the groupby object, not the order they are first observed. Parameters ---------- ascending : bool, default True If False, number in reverse, from number of group - 1 to 0. Returns ------- Series Unique numbers for each group. See Also -------- .cumcount : Number the rows in each group. Examples -------- >>> df = pd.DataFrame({"A": list("aaabba")}) >>> df A 0 a 1 a 2 a 3 b 4 b 5 a >>> df.groupby('A').ngroup() 0 0 1 0 2 0 3 1 4 1 5 0 dtype: int64 >>> df.groupby('A').ngroup(ascending=False) 0 1 1 1 2 1 3 0 4 0 5 1 dtype: int64 >>> df.groupby(["A", [1,1,2,3,2,1]]).ngroup() 0 0 1 0 2 1 3 3 4 2 5 0 dtype: int64 rrrrN) rrr+rxr$rrrrrrrr)rXrr+comp_idsrTrrYrYrZngroup> s = $zGroupBy.ngroupcCsR||j|j}|j|d}|||WdS1s"wYdS)a Number each item in each group from 0 to the length of that group - 1. Essentially this is equivalent to .. code-block:: python self.apply(lambda x: pd.Series(np.arange(len(x)), x.index)) Parameters ---------- ascending : bool, default True If False, number in reverse, from length of group - 1 to 0. Returns ------- Series Sequence number of each element within each group. See Also -------- .ngroup : Number the groups themselves. Examples -------- >>> df = pd.DataFrame([['a'], ['a'], ['a'], ['b'], ['b'], ['a']], ... columns=['A']) >>> df A 0 a 1 a 2 a 3 b 4 b 5 a >>> df.groupby('A').cumcount() 0 0 1 1 2 2 3 0 4 1 5 3 dtype: int64 >>> df.groupby('A').cumcount(ascending=False) 0 3 1 2 2 1 3 1 4 0 5 0 dtype: int64 )rN)rrrrsrr)rXrr+ cumcountsrYrYrZcumcount s 7  $zGroupBy.cumcountaveragekeepr na_optionpctc st|dvr d}t|||||ddkr.dd<fdd}|j||jd d }|S|j dd d S)a_ Provide the rank of values within each group. Parameters ---------- method : {'average', 'min', 'max', 'first', 'dense'}, default 'average' * average: average rank of group. * min: lowest rank in group. * max: highest rank in group. * first: ranks assigned in order they appear in the array. * dense: like 'min', but rank always increases by 1 between groups. ascending : bool, default True False for ranks by high (1) to low (N). na_option : {'keep', 'top', 'bottom'}, default 'keep' * keep: leave NA values where they are. * top: smallest rank if ascending. * bottom: smallest rank if descending. pct : bool, default False Compute percentage rank of data within each group. axis : int, default 0 The axis of the object over which to compute the rank. Returns ------- DataFrame with ranking of values within each group %(see_also)s Examples -------- >>> df = pd.DataFrame( ... { ... "group": ["a", "a", "a", "a", "a", "b", "b", "b", "b", "b"], ... "value": [2, 4, 2, 3, 5, 1, 2, 4, 1, 5], ... } ... ) >>> df group value 0 a 2 1 a 4 2 a 2 3 a 3 4 a 5 5 b 1 6 b 2 7 b 4 8 b 1 9 b 5 >>> for method in ['average', 'min', 'max', 'dense', 'first']: ... df[f'{method}_rank'] = df.groupby('group')['value'].rank(method) >>> df group value average_rank min_rank max_rank dense_rank first_rank 0 a 2 1.5 1.0 2.0 1.0 1.0 1 a 4 4.0 4.0 4.0 3.0 4.0 2 a 2 1.5 1.0 2.0 1.0 2.0 3 a 3 3.0 3.0 3.0 2.0 3.0 4 a 5 5.0 5.0 5.0 4.0 5.0 5 b 1 1.5 1.0 2.0 1.0 1.0 6 b 2 3.0 3.0 3.0 2.0 3.0 7 b 4 4.0 4.0 4.0 3.0 4.0 8 b 1 1.5 1.0 2.0 1.0 2.0 9 b 5 5.0 5.0 5.0 4.0 5.0 >toprbottomz3na_option must be one of 'keep', 'top', or 'bottom') ties_methodrrrrrrcs|jdddS)NF)rsrrYrankrrsr`rYrZrrzGroupBy.rank..TrrF)rrsNr)rpoprrr) rXrrrrrsrrarrYrrZr s.Hz GroupBy.rankcLtd|ddgdkrfdd}|j||jddS|jd iS) zq Cumulative product for each group. Returns ------- Series or DataFrame cumprodrrrc|jddiSNrsrYrrrrYrZr6rz!GroupBy.cumprod..TrNrnvvalidate_groupby_funcrrrrXrsr_r`rarYrrZr)  zGroupBy.cumprodcr) zm Cumulative sum for each group. Returns ------- Series or DataFrame rrrrcrrrrrrYrZrHrz GroupBy.cumsum..TrNrrrrYrrZr;rzGroupBy.cumsumc s`|dd}dkr(fdd}|d|}|j}|r |}|j||ddS|jd||d S) zm Cumulative min for each group. Returns ------- Series or DataFrame rTrctj|SrV)rminimum accumulaterrrYrZrZz GroupBy.cummin..cummaxrcumminrrrrWrrSrrrXrsrr`rrarrrrYrrZrM  zGroupBy.cumminc s`|dd}dkr(fdd}|d|}|j}|r |}|j||ddS|jd||dS) zm Cumulative max for each group. Returns ------- Series or DataFrame rTrcrrV)rmaximumrrrrYrZrrrz GroupBy.cummax..rrrrrrYrrZrerzGroupBy.cummax base_funcrnp.dtyperrrc  s,j} j| |dd} rtstdrtstd j} | j\} }t| dd f d d } j}|jd k} }t |}| rT| }|j |d d }|st |j |kr| dd}tt ||t |j dkr|j |dd td|r |}n||} |S)ap Get result for Cythonized functions. Parameters ---------- base_func : callable, Cythonized function to be called cython_dtype : np.dtype Type of the array that will be modified by the Cython call. numeric_only : bool, default True Whether only numeric datatypes should be computed needs_counts : bool, default False Whether the counts should be a part of the Cython call needs_mask : bool, default False Whether boolean mask needs to be part of the Cython call signature needs_nullable : bool, default False Whether a bool specifying if the input is nullable is part of the Cython call signature pre_processing : function, default None Function to be applied to `values` prior to passing to Cython. Function should return a tuple where the first element is the values to be passed to Cython and the second element is an optional type which the values should be converted to after being returned by the Cython operation. This function is also responsible for raising a TypeError if the values have an invalid type. Raises if `needs_values` is False. post_processing : function, default None Function to be applied to result of Cython function. Should accept an array of values as the first argument and type inferences as its second argument, i.e. the signature should be (ndarray, Type). If `needs_nullable=True`, a third argument should be `nullable`, to allow for processing specific to nullable values. **kwargs : dict Extra arguments to be passed back to Cython funcs Returns ------- `Series` or `DataFrame` with filled values rrz%'post_processing' must be a callable!z$'pre_processing' must be a callable!)r1rrrTc st|j}|jdkr dn|jd}tj|d}||f}t|d}d}r8tj jtjd}t||d}|}rB|\}}|j dd}|jdkrS|d}t||d}rtt | tj }|jdkrn|d d}t||d }rt |t}t||d }|di|jdkr|jddksJ|j|ddd f}ri} rt |t| d <||fi| }|jS)Nrr)r)r$Fr)rr)rr)r2)rrrrY)rrrrzerosrrrrrr1ruint8rr5) rrrr inferencesr$rr2 is_nullable pp_kwargs rrr`rrrrrrrXrYrZrksD            z0GroupBy._get_cythonized_result..blk_funcrTrgroup_FrNr)rbrWrrrxr$rrrrrrrrreplacerrrrrmrJ)rXrrrrrrrrr`rNrrxr^r5rkrrrrn orig_mgr_lenrohowstrrrYrrZr}s:4     2     zGroupBy._get_cythonized_resultc sdusdkrfdd}|j||jddS|jj\}}}tjt|tjd} t | |||j } | j |j | j |j | fidd} | S) u Shift each group by periods observations. If freq is passed, the index will be increased using the periods and the freq. Parameters ---------- periods : int, default 1 Number of periods to shift. freq : str, optional Frequency string. axis : axis to shift, default 0 Shift direction. fill_value : optional The scalar value to use for newly introduced missing values. Returns ------- Series or DataFrame Object shifted within each group. See Also -------- Index.shift : Shift values of Index. tshift : Shift the time index, using the index’s frequency if available. Nrcs|SrV)shiftrrsr freqperiodsrYrZr/rzGroupBy.shift..Trr)r  allow_dups)rrrxr$rrrrrgroup_shift_indexerr_reindex_with_indexersrsr&) rXrrrsr rar^r5r res_indexerrrrrYrrZrsz GroupBy.shiftrcsdkr|fddS|j}|jd}ddg|jdkr/|jvr+|d}||Sfd d |jD}t|rI|d d |D}||S) a First discrete difference of element. Calculates the difference of each element compared with another element in the group (default is element in previous row). Parameters ---------- periods : int, default 1 Periods to shift for calculating difference, accepts negative values. axis : axis to shift, default 0 Take difference over rows (0) or columns (1). Returns ------- Series or DataFrame First differences. rcs|jdS)Nrrs)rr)rsrrYrZrWrzGroupBy.diff..rrint16rfloat32csg|] \}}|vr|qSrYrY)rcrT) dtypes_to_f32rYrZrcsz GroupBy.diff..cSsi|]}|dqS)rrY)rrrYrYrZ ez GroupBy.diff..) rcrrrrTrdtypesrr)rXrrsrrshifted to_coercerY)rsrrrZr@s   z GroupBy.diffrsc sdusdkrfdd}|j||jddSdur#ddt|d}|j|jj|j|jd }|j|jd } || d S) z Calculate pct_change of each value to previous entry in group. Returns ------- Series or DataFrame Percentage changes within each group. Nrcs|jdS)N)r fill_methodrfrrs) pct_changerrsrrrfrrYrZrxsz$GroupBy.pct_change..Trrsrt)rsr~)rrrsr) rrrgrRrxcodesrsr~r) rXrrrfrrsrafilledfill_grprrYrrZris zGroupBy.pct_changecCs"||td|}||S)a Return first n rows of each group. Similar to ``.apply(lambda x: x.head(n))``, but it returns a subset of rows from the original DataFrame with original index and order preserved (``as_index`` flag is ignored). Parameters ---------- n : int If positive: number of entries to include from start of each group. If negative: number of entries to exclude from end of each group. Returns ------- Series or DataFrame Subset of original Series or DataFrame as determined by n. %(see_also)s Examples -------- >>> df = pd.DataFrame([[1, 2], [1, 4], [5, 6]], ... columns=['A', 'B']) >>> df.groupby('A').head(1) A B 0 1 2 2 5 6 >>> df.groupby('A').head(-1) A B 0 1 2 NrrslicerrXrr2rYrYrZheads# z GroupBy.headcCs4||r|t| d}n|g}||S)a Return last n rows of each group. Similar to ``.apply(lambda x: x.tail(n))``, but it returns a subset of rows from the original DataFrame with original index and order preserved (``as_index`` flag is ignored). Parameters ---------- n : int If positive: number of entries to include from end of each group. If negative: number of entries to exclude from start of each group. Returns ------- Series or DataFrame Subset of original Series or DataFrame as determined by n. %(see_also)s Examples -------- >>> df = pd.DataFrame([['a', 1], ['a', 2], ['b', 1], ['b', 2]], ... columns=['A', 'B']) >>> df.groupby('A').tail(1) A B 1 a 2 3 b 2 >>> df.groupby('A').tail(-1) A B 1 a 2 3 b 2 NrrrYrYrZtails $  z GroupBy.tailr2npt.NDArray[np.bool_]cCs@|jjd}||dk@}|jdkr|j|S|jjdd|fS)a Return _selected_obj with mask applied to the correct axis. Parameters ---------- mask : np.ndarray[bool] Boolean mask to apply. Returns ------- Series or DataFrame Filtered _selected_obj. rrN)rxr$rsrr)rXr2r^rYrYrZrs    zGroupBy._mask_selected_objr rc Cs|jj}t|dkr |S|jr|Stdd|Ds|Sdd|D}|jj}|dur5|||dg}tj||d \}}|j rX|j |j |dd d |i} |jdi| Sd dt|D} t| \} } |jt| dd }||jjj|d |d }|j| d}|jddS)aI If we have categorical groupers, then we might want to make sure that we have a fully re-indexed output to the levels. This means expanding the output space to accommodate all values in the cartesian product of our groups, regardless of whether they were observed in the data or not. This will expand the output space if there are missing groups. The method returns early without modifying the input if the number of groupings is less than 2, self.observed == True or none of the groupers are categorical. Parameters ---------- output : Series or DataFrame Object resulting from grouping and applying an operation. fill_value : scalar, default np.NaN Value to use for unobserved categories if self.observed is False. qs : np.ndarray[float64] or None, default None quantile values, only relevant for quantile. Returns ------- Series or DataFrame Object (potentially) re-indexed to include all possible groups. rcss |] }t|jttfVqdSrV)rgrouping_vectorr7rBrpingrYrYrZrs  z*GroupBy._reindex_output..cSsg|]}|jqSrY) group_indexrrYrYrZr%rz+GroupBy._reindex_output..N)rr Fr css$|] \}}|jr||jfVqdSrV)rre)rrhrrYrYrZrBs   )r1rs)r r )rvT)droprY)rxrrr|rrappendrD from_product sortlevelr{rr_get_axis_namersr.rrrr set_indexr"r() rXrAr rCr levels_listrr+r5d in_axis_grpsg_numsg_namesrYrYrZrIs>       zGroupBy._reindex_output int | Nonefrac float | NonerweightsSequence | Series | None random_stateRandomState | NonecCst|||}|durtj|j||jd}t|}|j|j|j}g} |D]9\} } |j | } t | } |dur;|}n |dusAJt || }tj| |||durRdn|| |d}| | |q't | } |jj| |jdS)a Return a random sample of items from each group. You can use `random_state` for reproducibility. .. versionadded:: 1.1.0 Parameters ---------- n : int, optional Number of items to return for each group. Cannot be used with `frac` and must be no larger than the smallest group unless `replace` is True. Default is one if `frac` is None. frac : float, optional Fraction of items to return. Cannot be used with `n`. replace : bool, default False Allow or disallow sampling of the same row more than once. weights : list-like, optional Default None results in equal probability weighting. If passed a list-like then values must have the same length as the underlying DataFrame or Series object and will be used as sampling probabilities after normalization within each group. Values must be non-negative with at least one positive element within each group. random_state : int, array-like, BitGenerator, np.random.RandomState, np.random.Generator, optional If int, array-like, or BitGenerator, seed for random number generator. If np.random.RandomState or np.random.Generator, use as given. .. versionchanged:: 1.4.0 np.random.Generator objects now accepted Returns ------- Series or DataFrame A new object of same type as caller containing items randomly sampled within each group from the caller object. See Also -------- DataFrame.sample: Generate random samples from a DataFrame object. numpy.random.choice: Generate a random sample from a given 1-D numpy array. Examples -------- >>> df = pd.DataFrame( ... {"a": ["red"] * 2 + ["blue"] * 2 + ["black"] * 2, "b": range(6)} ... ) >>> df a b 0 red 0 1 red 1 2 blue 2 3 blue 3 4 black 4 5 black 5 Select one row at random for each distinct value in column a. The `random_state` argument can be used to guarantee reproducibility: >>> df.groupby("a").sample(n=1, random_state=1) a b 4 black 4 2 blue 2 1 red 1 Set `frac` to sample fixed proportions rather than counts: >>> df.groupby("a")["b"].sample(frac=0.5, random_state=2) 5 5 2 2 0 0 Name: b, dtype: int64 Control sample probabilities within groups by setting weights: >>> df.groupby("a").sample( ... n=1, ... weights=[1, 1, 1, 0, 0, 1], ... random_state=1, ... ) a b 5 black 5 2 blue 2 0 red 0 Nr)r&rrr )sampleprocess_sampling_sizepreprocess_weightsrrsrr rxrrrroundrrrr-)rXrrrrr r& weights_arrgroup_iteratorsampled_indicesr1rr grp_indices group_size sample_size grp_samplerYrYrZr Ts2`       zGroupBy.sample NrNNNNTTTFFFT)rrrrtrrsrrvrorxrr}rrror{rrurr~rrzrr|rryrrwrrTrU)rirf)rerfrTr)rTrU)rTr)rTr)FF)rrrr)rrrTr)rr=rTr>rV)rArBrCrD)rAr=rTr>)rrrrrr)rNrfrrrsrrTr)rNrfrrrrrTrU)rrrhri)rTr)NFF) rarrLrrrrrrrrTr)Tr)rrrrrrfrr)rrrrrrrTr)rT) rNrfrrrrrrrr)Tr)rNrfrrrsr)rrrTr)T)rrrTr)rTr)rrrr)rr)rrrrfrhri)rr)rrrrrhrirr)rrrrr)rrrrrrrhri)rrrr)FrNN)rrrrrrrhri)Fr)rrrr)rTr;)rTrM)rTrK)rTrL)r`ra)rTrA)rr}rwr~rTr)rrfrr)rr)rTrFr) rrfrrrrfrrrsrrTrrC)rF) rrrrrrrrrrrr)rNrN)rr)rrrsrrTr)rrsNNr)r)rrrTr)r2rrTr)rArr rrCrDrTr)NNFNN) rrrrrrrrr r )arbrkrlrmrrr[rjrrrrrrr6rr@rJrKrMrWrXrgrrrwrzr" _apply_docsformatrcrrrrrrrrrrrrrr#_common_see_alsorr}rrrr rrrr r&r%_groupby_agg_method_templater)r/r0r6r=rFrHr;rQrXr\r^r_rqrsryrdr{r|rrrrrrrrrrrrrrrrrNaNrIr rYrYrYrZrSMs6 C : I  I 6! 5  "6H! (?- '.+TG93E:!  e T  l\2K:_.&$)^rSTFrrr<byrrsrrxrr{rrur~rrzr|ryrwrTcCsjt|trddlm}|}nt|trddlm}|}ntd|||||||||||| | | | | dS)Nr) SeriesGroupBy)DataFrameGroupByzinvalid type: )rrrtrsrvrxr}rr{rur~rzr|ryrw)rrGpandas.core.groupby.genericrr;rr)rrrrsrvrxr}rr{rur~rzr|ryrwrrrrYrYrZ get_groupbys.    r rrCrCnpt.NDArray[np.float64]rDcst||jr.N)rrr) r _is_multirrDrC factorizerrrrrrr)rrC lev_codeslevrrmirYr"rZrHs &rHrNrfrUcCsn|tjur|stjd|jd|d|dttddS|tjur5tjd|jd|dttddSdS)NzDropping invalid columns in rPzQ is deprecated. In a future version, a TypeError will be raised. Before calling .z=, select only columns which should be valid for the function.rz%The default value of numeric_only in z is deprecated. In a future version, numeric_only will default to False. Either specify numeric_only or select only columns which should be valid for the function.)rrrrrbrr&)clsrNrrYrYrZr!s.  rr)rrr<rrrsrrxrr{rrurr~rrzrr|rryrrwrrTrS)rrCrCr!rTrD)rNrfrTrU)rm __future__r contextlibrr functoolsrrrtextwraprrtypingrrr r r r r rrrrrrrnumpyrpandas._config.configr pandas._libsrrpandas._libs.groupby_libsrRrpandas._typingrrrrrrrrpandas.compat.numpyrr pandas.errorsr r!pandas.util._decoratorsr"r#r$r%pandas.util._exceptionsr&pandas.core.dtypes.castr'pandas.core.dtypes.commonr(r)r*r+r,r-r.r/r0pandas.core.dtypes.missingr1r2 pandas.corer3pandas.core._numbar4pandas.core.algorithmscorer(pandas.core.arraysr5r6r7r8pandas.core.baser9r:pandas.core.commoncommonrpandas.core.framer;pandas.core.genericr<pandas.core.groupbyr=r>r?pandas.core.groupby.indexingr@rApandas.core.indexes.apirBrCrDrEpandas.core.internals.blocksrFpandas.core.sampler pandas.core.seriesrGpandas.core.sortingrHpandas.core.util.numba_rIrJr[rKrLrMrrrr_transform_template _agg_templaterQ _KeysArgTypernrrSr rHrrYrYrYrZs   < (   ,        5A-3M    X# /