Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MINOR: [Python] fix pandas_compat.py #44171

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

Piong1997
Copy link

There is a bug that when column dtype is np.bytes,it will goto the final branch and run level=level.astype(dtype)

Rationale for this change

What changes are included in this PR?

Are these changes tested?

Are there any user-facing changes?

There is a bug that when column dtype is np.bytes,it will goto the final branch and run level=level.astype(dtype)
Copy link

Thanks for opening a pull request!

If this is not a minor PR. Could you open an issue for this pull request on GitHub? https://github.com/apache/arrow/issues/new/choose

Opening GitHub issues ahead of time contributes to the Openness of the Apache Arrow project.

Then could you also rename the pull request title in the following format?

GH-${GITHUB_ISSUE_ID}: [${COMPONENT}] ${SUMMARY}

or

MINOR: [${COMPONENT}] ${SUMMARY}

In the case of PARQUET issues on JIRA the title also supports:

PARQUET-${JIRA_ISSUE_ID}: [${COMPONENT}] ${SUMMARY}

See also:

@Piong1997 Piong1997 changed the title fix pandas_compat.py MINOR: [Python] fix pandas_compat.py Sep 19, 2024
@jorisvandenbossche
Copy link
Member

@Piong1997 do you have a reproducible example that shows the bug? (it would be good to add a test for it)

@Piong1997
Copy link
Author

@Piong1997 do you have a reproducible example that shows the bug? (it would be good to add a test for it)

Sure, the following code reproduce this bug:

import pandas as pd # pd.__version__ is 2.0.3
import pyarrow # pyarrow.__version__ is 17.0.0

df = pd.DataFrame([[0.1, 0.2], [0.3, 0.4]], columns=[b'col1', b'col2'])
table = pyarrow.Table.from_pandas(df)
table.to_pandas() # error occurred

Note that this bug can only be reproduced in pandas 2.0 or above.
This bug was introduced in a certain version of pyarrow, pyarrow version 6.0 can work normally.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants