EF Code First InvalidCastException attaching entity with multiple FK properties of same type

EF Code First InvalidCastException attaching entity with multiple FK properties of same type

I have a class with 2 properties of types derived from the same base class. When I try and DbSet.Attach to the entity, so I can mark it as modified, I get an InvalidCastException involving the two properties.
System.InvalidCastException: Unable to cast object of type
'DeliveryType' to type 'NotificationType'.
This is what my domain object looks like:
public class DomainObject
{
  ...
  [ForeignKey("DeliveryTypeObj")]
  public int DeliveryId { set; get; }

  [ForeignKey("NotificationTypeObj")]
  public int NotificationId { set; get; }

  public DeliveryType DeliveryTypeObj { set; get; }
  public NotificationType NotificationTypeObj { set; get; }
}
The two types 'DeliveryType' and 'NotificationType' are entities derived from a common base class that has just Id and Name properties (saves me from copying the same properties all over).
public class PickListObject
{
  public int Id { get; set; }
  public string Name { get; set; }
}

public class DeliveryType : PickListObject { /* Empty */ }
public class NotificationType : PickListObject { /* Empty */ }
I've seen that there is all sorts of wierdness when you have multiple properties of the same type in the same class, but those problems appear to be in the database creation, not in writing.
I could probably just get rid of the base class (PickListObject), and cut and paste the properties into each derived, but I have over 30 of them and that doesn't seem very clean.
I'd really appreciate any advice on how I can correct the exception so I can proceed.
Thanks!!!