The discrepancy between these two standard measures of inflation stems mostly from the differences between the basket of goods used by each index.
The CPI is based on survey data taken from some representative sample of consumers. This data represents a basket of "typical" household consumption. The PCE index is also based on survey data, but taken from firms instead. PCE index data represents a basket of sales derived from firm production. It is not difficult to imagine that this discrepancy in baskets can be magnified by certain prices, corresponding to goods in the difference set, receiving a much higher weight than others, thus driving a wedge between the indices. You allude to this in your question.
Additionally, and by construction, the CPI covers a smaller set of expenditures than those covered by the PCE index. The PCE index includes expenditures which are not paid for directly by the agent, such as medical care expenses from employer-provided health insurance, employee retirement payments, etc. The CPI does not take these sorts of transactions into account and instead covers strictly out-of-pocket expenses.
Lastly, the evolution of baskets characterized by the two indices over time is quite different. The CPI has a "stickier" basket that is much more resistant to quality-change and price substitution over time than the PCE index, which directly incorporates price substitution between goods in forming its representative basket.