fix Narrow the type of a when branching on a in b #2608#2627
Open
asukaminato0721 wants to merge 1 commit intofacebook:mainfrom
Open
fix Narrow the type of a when branching on a in b #2608#2627asukaminato0721 wants to merge 1 commit intofacebook:mainfrom
asukaminato0721 wants to merge 1 commit intofacebook:mainfrom
Conversation
There was a problem hiding this comment.
Pull request overview
Implements type narrowing for membership checks against mappings so that a in mapping (and the negated not in else-branch) can narrow a to the mapping’s key type, addressing issue #2608.
Changes:
- Add narrowing logic for
AtomicNarrowOp::Inwhen the RHS is aMapping[K, V], intersecting the LHS withK. - Add a regression test covering
not in dict[...]narrowing that enables safe indexing in the guarded branch.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
pyrefly/lib/alt/narrow.rs |
Adds mapping-aware narrowing for in by intersecting the LHS with the inferred mapping key type. |
pyrefly/lib/test/narrow.rs |
Adds a test ensuring not in registry correctly narrows the key type in the subsequent branch. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| // Check if the right operand is a mapping (e.g. dict[str, int]). | ||
| // If so, we can narrow the left operand to the mapping's key type. | ||
| if let Some((key_ty, _)) = self.unwrap_mapping(&right_ty) { | ||
| return self.intersect(ty, &key_ty); |
There was a problem hiding this comment.
unwrap_mapping intentionally returns Some for Any/types behaving like Any (see its doc comment). In that case key_ty becomes Any, and intersect(ty, Any) can widen the LHS to Any (since Any <: T), which is a regression for narrowing (it should never make the type less precise). Consider guarding this branch so it only narrows when the extracted key_ty does not behave like Any (e.g., if self.behaves_like_any(&key_ty) { ty.clone() } else { self.intersect(ty, &key_ty) }), or otherwise ensure intersection with Any preserves ty.
Suggested change
| return self.intersect(ty, &key_ty); | |
| // unwrap_mapping may return `Any` (or an `Any`-like type) for the key. | |
| // In that case, intersecting with `Any` could *widen* `ty` to `Any`, | |
| // which would be a regression for narrowing. If the key behaves like | |
| // Any, leave `ty` unchanged; otherwise, intersect with the key type. | |
| if self.behaves_like_any(&key_ty) { | |
| return ty.clone(); | |
| } else { | |
| return self.intersect(ty, &key_ty); | |
| } |
|
Diff from mypy_primer, showing the effect of this PR on open source code: python-chess (https://github.com/niklasf/python-chess)?target=https://github.com
- ERROR chess/engine.py:1333:17-1339:64: Pyrefly detected conflicting types while breaking a dependency cycle: `str | Any | None` is not assignable to `None`. Adding explicit type annotations might possibly help. [bad-assignment]
+ ERROR chess/engine.py:1333:17-1339:64: Pyrefly detected conflicting types while breaking a dependency cycle: `str | None` is not assignable to `None`. Adding explicit type annotations might possibly help. [bad-assignment]
kornia (https://github.com/kornia/kornia)?target=https://github.com
- ERROR kornia/models/efficient_vit/nn/act.py:43:39-43: Cannot index into `dict[str, type[Any]]` [bad-index]
poetry (https://github.com/python-poetry/poetry)?target=https://github.com
- ERROR tests/console/commands/self/test_show_plugins.py:125:51-56: Cannot index into `dict[str, list[Unknown]]` [bad-index]
antidote (https://github.com/Finistere/antidote)?target=https://github.com
+ ERROR tests/core/test_catalog.py:371:5-37: Object of class `Provider` has no attribute `data` [missing-attribute]
+ ERROR tests/core/test_catalog_test.py:44:9-52: Object of class `Provider` has no attribute `add` [missing-attribute]
+ ERROR tests/core/test_catalog_test.py:231:13-50: Object of class `Provider` has no attribute `data` [missing-attribute]
zulip (https://github.com/zulip/zulip)?target=https://github.com
- ERROR zerver/actions/message_send.py:1611:20-60: Argument `object` is not assignable to parameter `setting_group_id` with type `int` in function `zerver.lib.user_groups.check_any_user_has_permission_by_role` [bad-argument-type]
- ERROR zerver/actions/message_send.py:1614:41-81: Cannot index into `dict[int, str]` [bad-index]
- ERROR zerver/actions/message_send.py:1625:21-60: Argument `object` is not assignable to parameter `setting_group_id` with type `int` in function `zerver.lib.user_groups.check_user_has_permission_by_role` [bad-argument-type]
+ ERROR zerver/data_import/rocketchat.py:592:81-95: Argument `str` is not assignable to parameter `name_id` with type `int` in function `truncate_name` [bad-argument-type]
- ERROR zerver/lib/rest.py:105:35-48: Cannot index into `dict[str, object]` [bad-index]
- ERROR zerver/tests/test_user_groups.py:275:40-80: Cannot index into `dict[int, UserGroupMembersDict]` [bad-index]
apprise (https://github.com/caronc/apprise)?target=https://github.com
- ERROR apprise/plugins/email/base.py:411:38-54: Cannot index into `dict[str, dict[str, int]]` [bad-index]
- ERROR apprise/plugins/pagertree.py:215:36-42: Cannot index into `dict[str, str]` [bad-index]
core (https://github.com/home-assistant/core)?target=https://github.com
- ERROR homeassistant/components/alexa/capabilities.py:1167:47-53: Cannot index into `dict[str, str]` [bad-index]
- ERROR homeassistant/components/bang_olufsen/media_player.py:638:46-67: Cannot index into `dict[str, str]` [bad-index]
- ERROR homeassistant/components/deconz/device_trigger.py:717:62-74: Cannot index into `dict[str, dict[tuple[str, str], dict[str, int]]]` [bad-index]
- ERROR homeassistant/components/deconz/device_trigger.py:775:37-49: Cannot index into `dict[str, dict[tuple[str, str], dict[str, int]]]` [bad-index]
- ERROR homeassistant/components/homekit/type_sensors.py:443:39-51: Cannot index into `dict[str, SI]` [bad-index]
- ERROR homeassistant/components/homematic/entity.py:176:24-35: Cannot set item in `dict[str, Any]` [unsupported-operation]
- ERROR homeassistant/components/homematic/entity.py:181:31-42: Cannot index into `dict[str, Any]` [bad-index]
- ERROR homeassistant/components/huawei_lte/binary_sensor.py:137:17-32: Cannot index into `dict[str, str]` [bad-index]
- ERROR homeassistant/components/hue/v1/device_trigger.py:140:31-49: Cannot index into `dict[str, dict[tuple[str, str], dict[str, int]]]` [bad-index]
- ERROR homeassistant/components/hue/v1/device_trigger.py:191:37-49: Cannot index into `dict[str, dict[tuple[str, str], dict[str, int]]]` [bad-index]
- ERROR homeassistant/components/intent/timers.py:311:27-42: Cannot index into `dict[str, (TimerEventType, TimerInfo) -> None]` [bad-index]
- ERROR homeassistant/components/intent/timers.py:350:27-42: Cannot index into `dict[str, (TimerEventType, TimerInfo) -> None]` [bad-index]
- ERROR homeassistant/components/intent/timers.py:379:27-42: Cannot index into `dict[str, (TimerEventType, TimerInfo) -> None]` [bad-index]
- ERROR homeassistant/components/intent/timers.py:417:27-42: Cannot index into `dict[str, (TimerEventType, TimerInfo) -> None]` [bad-index]
- ERROR homeassistant/components/intent/timers.py:443:27-42: Cannot index into `dict[str, (TimerEventType, TimerInfo) -> None]` [bad-index]
- ERROR homeassistant/components/intent/timers.py:476:27-42: Cannot index into `dict[str, (TimerEventType, TimerInfo) -> None]` [bad-index]
- ERROR homeassistant/components/logbook/processor.py:365:55-65: Cannot index into `dict[EventType[Any] | str, tuple[str, (LazyEventPartialState) -> dict[str, Any]]]` [bad-index]
- ERROR homeassistant/components/modbus/entity.py:310:17-82: Cannot index into `dict[str, tuple[str, str] | tuple[str, None]]` [bad-index]
- ERROR homeassistant/components/modbus/entity.py:367:35-55: Argument `Unknown | None` is not assignable to parameter `address` with type `int` in function `homeassistant.components.modbus.modbus.ModbusHub.async_pb_call` [bad-argument-type]
- ERROR homeassistant/components/mqtt/config_flow.py:965:64-75: Cannot index into `dict[SensorStateClass, set[str | type[StrEnum] | None]]` [bad-index]
- ERROR homeassistant/components/mqtt/tag.py:184:60-74: Cannot index into `dict[str, dict[str, MQTTTagScanner]]` [bad-index]
- ERROR homeassistant/components/mqtt/tag.py:185:22-36: Cannot index into `dict[str, dict[str, MQTTTagScanner]]` [bad-index]
- ERROR homeassistant/components/number/__init__.py:472:52-64: Cannot index into `dict[NumberDeviceClass, type[BaseUnitConverter]]` [bad-index]
- ERROR homeassistant/components/number/__init__.py:496:36-48: Cannot index into `dict[NumberDeviceClass, type[BaseUnitConverter]]` [bad-index]
- ERROR homeassistant/components/number/__init__.py:513:32-44: Cannot index into `dict[NumberDeviceClass, type[BaseUnitConverter]]` [bad-index]
- ERROR homeassistant/components/number/__init__.py:514:48-60: Cannot index into `dict[NumberDeviceClass, type[BaseUnitConverter]]` [bad-index]
- ERROR homeassistant/components/ps4/media_player.py:205:54-75: Cannot index into `dict[str, bool | dict[str, JsonValueType] | float | int | list[JsonValueType] | str | None]` [bad-index]
- ERROR homeassistant/components/ps4/media_player.py:289:54-75: Cannot index into `dict[str, bool | dict[str, JsonValueType] | float | int | list[JsonValueType] | str | None]` [bad-index]
- ERROR homeassistant/components/ps4/media_player.py:295:32-55: No matching overload found for function `dict.pop` called with arguments: (str | None) [no-matching-overload]
- ERROR homeassistant/components/reolink/__init__.py:549:32-48: Cannot index into `dict[str, int]` [bad-index]
- ERROR homeassistant/components/screenlogic/coordinator.py:39:36-39: Cannot index into `dict[str, dict[str, Any]]` [bad-index]
- ERROR homeassistant/components/stream/__init__.py:397:27-40: Cannot index into `dict[str, StreamOutput]` [bad-index]
- ERROR homeassistant/components/stream/__init__.py:398:31-44: Cannot delete item in `dict[str, StreamOutput]` [unsupported-operation]
- ERROR homeassistant/components/subaru/sensor.py:262:67-70: Cannot index into `dict[str, Unknown]` [bad-index]
- ERROR homeassistant/components/subaru/sensor.py:265:75-78: Cannot index into `dict[str, Unknown]` [bad-index]
- ERROR homeassistant/components/synology_dsm/services.py:59:34-40: Cannot index into `dict[str, SynologyDSMData]` [bad-index]
- ERROR homeassistant/helpers/entity_registry.py:1393:61-76: Cannot index into `dict[str, set[str | None]]` [bad-index]
+ ERROR homeassistant/helpers/singleton.py:59:33-52: Cannot index into `HassDict` [bad-index]
+ ERROR homeassistant/helpers/singleton.py:73:26-45: Cannot index into `HassDict` [bad-index]
+ ERROR homeassistant/helpers/singleton.py:77:33-52: Cannot index into `HassDict` [bad-index]
zope.interface (https://github.com/zopefoundation/zope.interface)?target=https://github.com
- ERROR src/zope/interface/tests/test_odd_declarations.py:221:30-57: `-` is not supported between `None` and `type[I2]` [unsupported-operation]
operator (https://github.com/canonical/operator)?target=https://github.com
- ERROR ops/model.py:2144:80-83: Cannot index into `Self@RelationDataContent` [bad-index]
aiortc (https://github.com/aiortc/aiortc)?target=https://github.com
+ ERROR src/aiortc/rtcpeerconnection.py:1101:27-44: Object of class `RTCRtpTransceiver` has no attribute `start` [missing-attribute]
- ERROR src/aiortc/rtcpeerconnection.py:1102:25-46: Argument `RTCSctpCapabilities | None` is not assignable to parameter `remoteCaps` with type `RTCSctpCapabilities` in function `aiortc.rtcsctptransport.RTCSctpTransport.start` [bad-argument-type]
- ERROR src/aiortc/rtcpeerconnection.py:1102:48-69: Argument `int | None` is not assignable to parameter `remotePort` with type `int` in function `aiortc.rtcsctptransport.RTCSctpTransport.start` [bad-argument-type]
- ERROR src/aiortc/rtcsctptransport.py:1673:21-35: `+=` is not supported between `None` and `Literal[2]` [unsupported-operation]
dd-trace-py (https://github.com/DataDog/dd-trace-py)?target=https://github.com
- ERROR ddtrace/appsec/_iast/taint_sinks/ast_taint.py:46:58-84: `+` is not supported between `LiteralString` and `None` [unsupported-operation]
- ERROR ddtrace/internal/core/event_hub.py:68:24-32: Cannot set item in `dict[str, dict[Any, (...) -> Any]]` [unsupported-operation]
- ERROR ddtrace/internal/core/event_hub.py:68:73-81: Cannot index into `dict[str, dict[Any, (...) -> Any]]` [bad-index]
- ERROR ddtrace/internal/coverage/code.py:356:52-68: Cannot index into `dict[str, ModuleCodeCollector.CollectInContext]` [bad-index]
- ERROR ddtrace/internal/coverage/code.py:359:30-46: Cannot index into `dict[str, Unknown]` [bad-index]
- ERROR ddtrace/internal/coverage/code.py:360:43-59: Cannot index into `defaultdict[str, Unknown]` [bad-index]
- ERROR ddtrace/internal/coverage/code.py:360:82-98: Cannot index into `dict[str, Unknown]` [bad-index]
- ERROR ddtrace/internal/coverage/code.py:362:44-60: Cannot delete item in `dict[str, ModuleCodeCollector.CollectInContext]` [unsupported-operation]
- ERROR ddtrace/internal/coverage/code.py:369:52-68: Cannot index into `dict[str, ModuleCodeCollector.CollectInContext]` [bad-index]
- ERROR ddtrace/internal/coverage/code.py:372:30-46: Cannot index into `dict[str, Unknown]` [bad-index]
- ERROR ddtrace/internal/coverage/code.py:373:43-59: Cannot index into `defaultdict[str, Unknown]` [bad-index]
- ERROR ddtrace/internal/coverage/code.py:373:82-98: Cannot index into `dict[str, Unknown]` [bad-index]
- ERROR ddtrace/internal/coverage/code.py:375:44-60: Cannot delete item in `dict[str, ModuleCodeCollector.CollectInContext]` [unsupported-operation]
pydantic (https://github.com/pydantic/pydantic)?target=https://github.com
- ERROR pydantic/v1/generics.py:124:25-132:14: No matching overload found for function `pydantic.v1.main.create_model` called with arguments: (str, __module__=str, __base__=tuple[type[GenericModelT], ...], __config__=None, __validators__=dict[str, classmethod[Any, Ellipsis, Any] | classmethod[Any, Ellipsis, Unknown]], __cls_kwargs__=None, **dict[Unknown, tuple[DeferredType, FieldInfo]]) [no-matching-overload]
+ ERROR pydantic/v1/generics.py:124:25-132:14: No matching overload found for function `pydantic.v1.main.create_model` called with arguments: (str, __module__=str, __base__=tuple[type[GenericModelT], ...], __config__=None, __validators__=dict[str, classmethod[Any, Ellipsis, Any] | classmethod[Any, Ellipsis, Unknown]], __cls_kwargs__=None, **dict[str, tuple[DeferredType, FieldInfo]]) [no-matching-overload]
cloud-init (https://github.com/canonical/cloud-init)?target=https://github.com
- ERROR cloudinit/config/cc_ca_certs.py:118:44-120:6: No matching overload found for function `posixpath.join` called with arguments: (list[str] | str | None, list[str] | str | None) [no-matching-overload]
+ ERROR cloudinit/config/cc_ca_certs.py:118:44-120:6: No matching overload found for function `posixpath.join` called with arguments: (list[str] | str | Unknown | None, list[str] | str | Unknown | None) [no-matching-overload]
- ERROR cloudinit/config/cc_resizefs.py:267:5-35: Cannot unpack tuple[str | Unknown, str | Any, str | Any] | tuple[str | Unknown, str | Any, str | Any, str] | tuple[str, str, str] | tuple[Unknown, Unknown, Unknown] | tuple[Unknown, Unknown, Unknown, Unknown] (of size 4) into 3 values [bad-unpacking]
+ ERROR cloudinit/config/cc_resizefs.py:267:5-35: Cannot unpack tuple[str | Unknown, str | Any, Unknown] | tuple[str | Unknown, str | Any, Unknown, str] | tuple[str, str, str] | tuple[Unknown, Unknown, Unknown] | tuple[Unknown, Unknown, Unknown, Unknown] (of size 4) into 3 values [bad-unpacking]
- ERROR cloudinit/distros/opensuse.py:199:17-47: Cannot unpack tuple[str | Unknown, str | Any, str | Any] | tuple[str | Unknown, str | Any, str | Any, str] | tuple[str, str, str] | tuple[Unknown, Unknown, Unknown] | tuple[Unknown, Unknown, Unknown, Unknown] (of size 4) into 3 values [bad-unpacking]
+ ERROR cloudinit/distros/opensuse.py:199:17-47: Cannot unpack tuple[str | Unknown, str | Any, Unknown] | tuple[str | Unknown, str | Any, Unknown, str] | tuple[str, str, str] | tuple[Unknown, Unknown, Unknown] | tuple[Unknown, Unknown, Unknown, Unknown] (of size 4) into 3 values [bad-unpacking]
- ERROR cloudinit/sources/DataSourceVMware.py:995:24-31: Cannot index into `dict[str, dict[str, Interface]]` [bad-index]
- ERROR cloudinit/sources/DataSourceVMware.py:1003:24-31: Cannot index into `dict[str, dict[str, Interface]]` [bad-index]
paasta (https://github.com/yelp/paasta)?target=https://github.com
+ ERROR paasta_tools/paastaapi/model_utils.py:213:20-44: Expected second argument to `super` to be a class object or instance, got `type[Self@OpenApiModel] | Unknown` [invalid-argument]
jinja (https://github.com/pallets/jinja)?target=https://github.com
- ERROR src/jinja2/lexer.py:167:34-44: Cannot index into `dict[LiteralString, str]` [bad-index]
mitmproxy (https://github.com/mitmproxy/mitmproxy)?target=https://github.com
- ERROR mitmproxy/proxy/events.py:92:46-57: Cannot index into `dict[Command, type[CommandCompleted]]` [bad-index]
bokeh (https://github.com/bokeh/bokeh)?target=https://github.com
- ERROR src/bokeh/document/callbacks.py:308:51-56: Cannot index into `dict[str, type[Event]]` [bad-index]
- ERROR src/bokeh/document/callbacks.py:315:42-47: Cannot index into `dict[str, list[Callback]]` [bad-index]
- ERROR src/bokeh/document/callbacks.py:318:39-44: Cannot index into `dict[str, list[(Event) -> None]]` [bad-index]
- ERROR src/bokeh/plotting/_graph.py:73:39-45: Argument `dict[Unknown, Unknown] | Unknown` is not assignable to parameter `field` with type `str` in function `bokeh.core.property.vectorization.Field.__init__` [bad-argument-type]
+ ERROR src/bokeh/plotting/_graph.py:73:39-45: Argument `dict[Unknown, Unknown] | str` is not assignable to parameter `field` with type `str` in function `bokeh.core.property.vectorization.Field.__init__` [bad-argument-type]
- ERROR src/bokeh/plotting/_renderer.py:217:46-51: Cannot index into `dict[str, Any]` [bad-index]
- ERROR src/bokeh/plotting/_renderer.py:225:43-48: Cannot index into `dict[str, Any]` [bad-index]
static-frame (https://github.com/static-frame/static-frame)?target=https://github.com
- ERROR static_frame/core/util.py:2774:31-32: Cannot set item in `dict[TLabel, int]` [unsupported-operation]
- ERROR static_frame/core/util.py:2774:52-53: Cannot index into `dict[TLabel, int]` [bad-index]
- ERROR static_frame/core/util.py:2776:26-27: Cannot set item in `dict[TLabel, int]` [unsupported-operation]
mypy (https://github.com/python/mypy)?target=https://github.com
- ERROR mypy/checker.py:5959:89-93: Cannot index into `dict[SymbolNode, Type]` [bad-index]
- ERROR mypy/suggestions.py:194:36-44: Cannot index into `dict[SymbolNode, list[Type]]` [bad-index]
prefect (https://github.com/PrefectHQ/prefect)?target=https://github.com
- ERROR src/prefect/server/api/server.py:664:26-35: Cannot index into `dict[tuple[Settings, bool], Unknown]` [bad-index]
- ERROR src/prefect/server/events/actions.py:595:59-63: Cannot index into `dict[str, tuple[type[PrefectBaseModel], list[(...) -> Coroutine[Any, Any, Unknown]]]]` [bad-index]
pip (https://github.com/pypa/pip)?target=https://github.com
- ERROR src/pip/_internal/operations/prepare.py:627:42-50: Cannot index into `dict[str, str]` [bad-index]
optuna (https://github.com/optuna/optuna)?target=https://github.com
- ERROR optuna/storages/_cached_storage.py:95:67-75: Cannot delete item in `dict[int, tuple[int, int]]` [unsupported-operation]
schema_salad (https://github.com/common-workflow-language/schema_salad)?target=https://github.com
- ERROR schema_salad/avro/schema.py:207:27-31: Cannot index into `dict[str, NamedSchema]` [bad-index]
meson (https://github.com/mesonbuild/meson)?target=https://github.com
- ERROR mesonbuild/backend/vs2010backend.py:1046:27-28: Cannot index into `dict[Language, CompilerArgs]` [bad-index]
- ERROR mesonbuild/backend/vs2010backend.py:1046:27-28: Cannot set item in `dict[Language, CompilerArgs]` [unsupported-operation]
cwltool (https://github.com/common-workflow-language/cwltool)?target=https://github.com
+ WARN cwltool/builder.py:316:46-67: Redundant cast: `str` is the same type as `str` [redundant-cast]
- ERROR cwltool/cwlprov/ro.py:321:42-51: Cannot index into `dict[str, str]` [bad-index]
- ERROR cwltool/cwlprov/ro.py:327:51-60: Cannot index into `dict[str, str]` [bad-index]
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes #2608
Implemented narrowing for a in mapping by intersecting the LHS with the mapping key type. This fixes not in branches by negation.
Test Plan
add test