Choose by state
CANCEL attempts to stop a pending request, most commonly a ringing INVITE. BYE terminates an established session within a dialog. Sending CANCEL does not undo an INVITE that already completed successfully. This is why a trace must be read in time order, with transaction and dialog identifiers, rather than by method names alone. A typical canceled ringing attempt contains a CANCEL response and a separate final response to the original INVITE. Those responses answer different requests.
Simplified caller-side decision. The CANCEL and BYE requests also receive their own responses; a successful answer racing with cancellation still needs ACK and dialog cleanup.
The normal cancellation sequence
After a provisional response, the caller sends CANCEL for the pending INVITE. A matching server normally acknowledges CANCEL with 200 OK and ends the INVITE with 487 Request Terminated. The caller then acknowledges that non-2xx INVITE response with ACK. The 200 response to CANCEL does not itself mean that an answered call was hung up. Check the CSeq method on every response. If the INVITE succeeded before cancellation took effect, the caller must handle the successful response, send ACK and then terminate the resulting dialog with BYE when appropriate.
Diagnosing an unexpected hangup
For an established call, find the first BYE and identify its sender on that leg. Then correlate the PBX's other leg, application logs and carrier evidence. A BYE from a PBX may reflect a remote hangup, a timeout or local application policy; it does not by itself reveal the original cause. An unsuccessful BYE response such as 481 suggests the receiving peer cannot match the dialog. Examine Call-ID and both tags, route changes, failover and state loss. Preserve a short trace around the transition instead of assuming every 487 is a carrier failure.
Sources & applicability
Primary references for the technical details above. Operational examples and planning checklists are VoIP.info editorial guidance.
Examples require adaptation to your topology. No live PBX or hardware testing is claimed.