Wsrtjtyk

A recent session saw us—the PCs—encounter a gate made of magical fire. The wall surrounding the gate was 30 ft. tall.

When the Oath of the Crown paladin and I were on one side of the gate, the paladin used on an enemy creature his champion challenge Channel Divinity option that says that the affected creature can't willingly move more than 30 ft. away from the paladin. Then my PC knocked the flying creature through the fire.

The DM tried to have the enemy creature fly over the fire to avoid having it go back through the fire, but I said that since up was more than 30 ft. away from the paladin the enemy creature couldn't fly that way.

The DM reluctantly agreed, but now I'm wondering who was right.

Should effects like champion challenge be considered spheres or planes?

Spells affect the space they are described to affect. In the case of an ability like Champion's Challenge, it's a sphere (3D).

From the description of spell (in this case ability) area of effects:

Every area of effect has a point of origin, a location from which the spell's energy erupts. The rules for each shape specify how you position its point of origin. Typically, a point of origin is a point in space, but some spells have an area whose origin is a creature or an object.

The Champion's Challenge ability specifies a range (radius) which means it's effectively a sphere because a creature may occupy a position anywhere in space around the paladin so long as it's not too far away, as measured from the paladin's body, not a specific point. If it were 2D, it would say something to the affect that it was just an area on the ground (like Spike Growth):

The ground in a 20-foot radius becomes difficult terrain for the duration [...]

If you count 30' from the top of the paladin's head, then there's a gap above the wall that is equal to the paladin's height. The flying creature could fly over the wall through this gap if it is able to fit.

This assumes the paladin is right next to the wall. If he's further away, the gap would rapidly narrow to nonexistence and eliminate a safe path for the flyer.

You are correct based on spell text. A grid battle map is 2d and is mainly used to make quick move / range calculations.

SPHERE (PHB pg. 204-205)

You select a sphere's point of origin, and the sphere extends outward from that point. The sphere's size is expressed as a radius in feet that extends from the point.

It’s 3d, but isn’t applicable until you encounter it in 3d space. Most of the time a template or square counting is sufficient, but like in your situation, it still effected what’s above you the same as ground level in front of you.

This is largely up to DM discretion

By RAW, the flying creature could only willingly fly 30 ft high if it was directly over the Paladin. In your example, 30 ft away and 30 ft high would be about 45 ft away from the paladin. However, many DMs will hand wave or approximate the vertical component to avoid messy math and arguments about how high is the center.

Now comes the DM fiat portion. Obviously, you can knock the creature out of the 30 ft range since this is unwilling movement. The difficulty comes in whether or not changing trajectory constitutes willing movement.

Personally, I would rule that willingly changing trajectory to fly over the portal constitutes willing movement. I would also require an Athletics (Dexterity) check to attempt it.