Use correct Cache-Tag (CDN) and X-RTD-Project header on subprojects

[humitos]

Currently, when accessing a subproject (eg. edx-guide-for-students being
subproject of edx) we are generating Cache-Tag: edx,edx-latest. Then, when
a build for edx-guide-for-students on version latest succeeds, we are purging
the cache tags edx-guide-for-students-latest.

This commit makes the Cache-Tag generated for subprojects to use the
subproject's slug instead of the parent's/superproject's slug and matching the
cache tag that we are purging when the build succeeds.

Besides, it fixes the X-RTD-Project header as well, since it was using the
same parent's/superproject's slug as value.

Fixes #7590

[humitos]

This is not related to this PR, but I wanted to make a note here, tho. Maybe a TODO where we set this header is better?

readthedocs.org/readthedocs/proxito/middleware.py

Lines 141 to 146 in a81d82e

	if hasattr(request, 'rtdheader'):
	response['X-RTD-Project-Method'] = 'rtdheader'
	elif hasattr(request, 'subdomain'):
	response['X-RTD-Project-Method'] = 'subdomain'
	elif hasattr(request, 'cname'):
	response['X-RTD-Project-Method'] = 'cname'

This header is wrong to all the subprojects when they are accessed via the domain of their parent's, eg. project.readthedocs.io/projects/subproject/en/latest/

[ericholscher]

Do we not need to use host_project_slug sometimes as well? It seems like this is breaking existing logic, unless path_project_slug is getting set other places?

[humitos]

Do we not need to use host_project_slug sometimes as well?

As a fallback, you mean? In case that path_project_slug is not available? Why it would not be available?

It seems like this is breaking existing logic, unless path_project_slug is getting set other places?

What existing logic do you think it's breaking? If you have an example on mind I can try to write a test case.

[humitos]

We are using the same logic for version_path_project that follows this line. It's used here and set in _get_project_data_from_request (

readthedocs.org/readthedocs/proxito/views/utils.py

Line 96 in a81d82e

request.path_version_slug = version_slug

) which is the first function that is called when serving a page at

readthedocs.org/readthedocs/proxito/views/serve.py

Lines 72 to 79 in a81d82e

	final_project, lang_slug, version_slug, filename = _get_project_data_from_request( # noqa
	request,
	project_slug=project_slug,
	subproject_slug=subproject_slug,
	lang_slug=lang_slug,
	version_slug=version_slug,
	filename=filename,
	)

The host_project_slug is returned by map_host_project_slug which is accurate,

readthedocs.org/readthedocs/proxito/middleware.py

Line 159 in a81d82e

ret = map_host_to_project_slug(request)

. However, the project slug we get from the host sometimes (in subprojects or translations, for example) is not the project slug being served.

[ericholscher]

Gotcha, are other uses of host_project_slug unaffected by this issue? It just seems bad to have 2 different variables for the project_slug, where we use them in different places.

[humitos]

It just seems bad to have 2 different variables for the project_slug, where we use them in different places.

Well, if I'm understand correctly this they are different things:

• host_project_slug is the project slug that we infer from the host/domain's URL
  • not accurate for subdomain/translations
  • it's fine for custom domains and .readthedocs.io
  • it's set at the beginning of the request in the middleware
• path_project_slug is the project slug that we can infer from the path's URL mainly
  • it's good for subdomain/translations
  • it's the same as what we have been calling final_project (maybe it's better to call it final_project_slug?)
  • it's set in the _get_project_data_from_request which is called from inside the ServeDocs view in Proxito as the first step of the request after all the middlewares

It seems that both are required. We use host_project_slug as first step trying to get the Project object (map_project_slug decorator). Then, inside the _get_project_data_from_request we use host_project_slug to check single project, translation and subproject and set the path_project_slug which in those cases could be different than host_project_slug.

Gotcha, are other uses of host_project_slug unaffected by this issue?

Grepping the code I didn't find anything that clearly needs to be adapted. Maybe there is something deeper that I'm not seeing, tho

[ericholscher]

This makes sense to me. We should probably document all these variables (as you have done above) in the code/docs, so we don't get confused in the future.

[ericholscher]

This makes sense to me, though we should audit to make sure we aren't creating additional bugs by not using the same method of resolving the project_slug everywhere.

[ericholscher]

Gonna merge this, but @humitos please check the other places this might be needed.

[nedbat]

Thanks so much, I love the responsiveness!