Skip to content
This repository was archived by the owner on Apr 12, 2024. It is now read-only.

Track Firebase hosting migration #16417

Closed
14 of 17 tasks
petebacondarwin opened this issue Jan 20, 2018 · 29 comments
Closed
14 of 17 tasks

Track Firebase hosting migration #16417

petebacondarwin opened this issue Jan 20, 2018 · 29 comments

Comments

@petebacondarwin
Copy link
Contributor

petebacondarwin commented Jan 20, 2018

@IgorMinar
Copy link
Contributor

@petebacondarwin thank you!

@IgorMinar
Copy link
Contributor

@Narretz is it correct that the timeout issue is resolved?

@Narretz
Copy link
Contributor

Narretz commented Jan 22, 2018

@IgorMinar yes the issue is resolved. It was an encoding issue not a file size issue. The zip files were additional gzipped for faster upload but this confused gcs / Firebase. I replaced all the available zips with their default form and now it works. The deployment has been updated too

@IgorMinar
Copy link
Contributor

So I guess that means that we are good to go then. Right?

I book Wed 9-10am PT on my calendar to make this go live + sent out an invite to you two.

@IgorMinar
Copy link
Contributor

@petebacondarwin what about blog.angularjs.org? have you thought about it? if we don't change anything, I believe that it will break because of the certs and the current proxy setup.

I think we'll need to do something like:

@petebacondarwin
Copy link
Contributor Author

@IgorMinar - yes we need to setup the DNS to CNAME to point to blogger and then set up the blog on Blogger to accept incoming connections from that domain - I believe there is now a setting that allows HTTPS. I don't have admin rights to do either of these two.

The alternative is to change blog.angularjs.org to point directly to blog.angular.io, which is not hosted on blogger but something else...

@IgorMinar
Copy link
Contributor

IgorMinar commented Jan 24, 2018

I was thinking about this more and I think the lowest risk will be to just setup a redirect from blog.angularjs.org to blogger. The current DNS setup is messed up anyway and most urls on the web point to either https://blog.angularjs.org or http://angularjs.blogspot.com/<somepath>.html.

@petebacondarwin
Copy link
Contributor Author

@IgorMinar that would be fine by me but I think it will need, yet another, Firebase project setup for the subdomain, right?

@IgorMinar
Copy link
Contributor

all but ci.angularjs.org are switched now. @petebacondarwin we need to decide what to do with the jenkins instance. without the global proxy we no longer have a signed certificate for it. Could you look into getting a cert from lets encrypt and installing it on the box?

@IgorMinar
Copy link
Contributor

haha.. so much for "we need to decide". above is my proposal. the only two alternatives is to use a self-signed cert (bad ux) or switch to http (bad security). @petebacondarwin if there are other options please propose them.

@petebacondarwin
Copy link
Contributor Author

Can we continue to use the IP address directly for Jenkins, @IgorMinar?

@IgorMinar
Copy link
Contributor

@petebacondarwin we can. but that's far from ideal :)

I update the DNS to point to the new ip address which uses self-signed cert. You can only bypass the cert if you visit the domain with a browser that previously hasn't visited the site because of angularjs.org previously serving Strict-Transport-Security: max-age=16070400; includeSubDomains; preload header.

So we'll either fix the cert or wait half a year for us to be able to bypass the cert warning.

If you don't care, we can just document this and move on...

@IgorMinar
Copy link
Contributor

I updated the hosting doc. let's move on.. :)

@IgorMinar
Copy link
Contributor

@IgorMinar that would be fine by me but I think it will need, yet another, Firebase project setup for the subdomain, right?

@petebacondarwin was this referring to dashboard.angularjs.org or something else?

@petebacondarwin
Copy link
Contributor Author

@IgorMinar - I was referring to setting up a "redirect" from blog.angularjs.org to blogger. A HTTP redirect (rather than a DNS based solution) would require a configured server.

@petebacondarwin
Copy link
Contributor Author

If you don't care, we can just document this and move on...

I don't care :-)

@IgorMinar
Copy link
Contributor

IgorMinar commented Jan 26, 2018 via email

@petebacondarwin
Copy link
Contributor Author

I have disabled the following jobs on ci.angularjs.org:

  • angularjs.org - this was building and publishing the home page when it changed
  • angular-sites - this was building and updating the nginx site config when it changed

@Narretz
Copy link
Contributor

Narretz commented Jan 29, 2018

Something that's not in the list is that we are no longer publishing the build artefacts to the code.angularjs.org github repository, since we deploy to Firebase directly to Travis. I mentioned this before, but I just wanted to make sure that we are on the same page about this.

@IgorMinar
Copy link
Contributor

there issues with code.angularjs.org robots.txt:

The following urls/patterns are not configured correctly:

@IgorMinar
Copy link
Contributor

IgorMinar commented Feb 4, 2018

I updated the checklist above with the summary of new issues that need followup.

@Narretz
Copy link
Contributor

Narretz commented Feb 5, 2018

@IgorMinar

  • you mentioned that we need to allow https://code.angularjs.org/snapshot/docs/js/all-versions-data.js in the code.angularjs.org folders, but do we actually want to index the versions on code? Don't we just want to index docs.angularjs.org? (which probably needs the same fix)
  • I assume it's not only all-versions-data.js (the first file alphabetically), it's all javascript assets?
  • wrt the sitemap.xml, we haven't had one for a long time, see Docs need sitemap.xml #8740. Obviously now is a good time to add one. @petebacondarwin is there something in dgeni that we can use to create a sitemap?

@Narretz
Copy link
Contributor

Narretz commented Feb 5, 2018

I've copied the sitemap generation script from angular.io, and it looks good. I just need to filter out the example files.

@petebacondarwin
Copy link
Contributor Author

I don't think we want to index any of code.angularjs.org

@Narretz
Copy link
Contributor

Narretz commented Feb 5, 2018

Yeah, I realized versions-data.js is needed because it is loaded by docs.angularjs.org, so that it always has the most up to date list of versions.

@IgorMinar
Copy link
Contributor

IgorMinar commented Feb 5, 2018 via email

@Narretz
Copy link
Contributor

Narretz commented Feb 5, 2018

The robots.txt at code and docs have been updated:
https://docs.angularjs.org/robots.txt
https://code.angularjs.org/robots.txt

Sitemap has been added to docs:
https://docs.angularjs.org/sitemap.xml

@IgorMinar
Copy link
Contributor

IgorMinar commented Feb 6, 2018 via email

@petebacondarwin
Copy link
Contributor Author

If we allow the partials then they become indexed, which means that the raw partial is visible in searches. I don't think it matters what we do with images - we should probably allow them so that people can search for images in the guides.

Narretz added a commit to Narretz/angular.js that referenced this issue Feb 12, 2018
This commit restores serving the plain partials (content) when a docs
page is accessed with ?_escaped_fragment_=.
The Google Ajax Crawler accesses these urls when the page has
`<meta type="fragment" content="!">` is set.

During the migration to Firebase, this was lost, which resulted in Google
dropping the docs almost completely from the index.

We are using a Firebase cloud function to serve the partials. Since
we cannot access the static hosted files from the function, we have to
deploy them as part of the function directory instead, from which they
can be read.

Related to angular#16432
Related to angular#16417
Narretz added a commit to Narretz/angular.js that referenced this issue Feb 12, 2018
This commit restores serving the plain partials (content) when a docs
page is accessed with ?_escaped_fragment_=.
The Google Ajax Crawler accesses these urls when the page has
`<meta type="fragment" content="!">` is set.

During the migration to Firebase, this was lost, which resulted in Google
dropping the docs almost completely from the index.

We are using a Firebase cloud function to serve the partials. Since
we cannot access the static hosted files from the function, we have to
deploy them as part of the function directory instead, from which they
can be read.

Related to angular#16432
Related to angular#16417
Narretz added a commit to Narretz/angular.js that referenced this issue Feb 12, 2018
This commit restores serving the plain partials (content) when a docs
page is accessed with ?_escaped_fragment_=.
The Google Ajax Crawler accesses these urls when the page has
`<meta type="fragment" content="!">` is set.

During the migration to Firebase, this was lost, which resulted in Google
dropping the docs almost completely from the index.

We are using a Firebase cloud function to serve the partials. Since
we cannot access the static hosted files from the function, we have to
deploy them as part of the function directory instead, from which they
can be read.

Related to angular#16432
Related to angular#16417
Narretz added a commit to Narretz/angular.js that referenced this issue Feb 12, 2018
This commit restores serving the plain partials (content) when a docs
page is accessed with ?_escaped_fragment_=.
The Google Ajax Crawler accesses these urls when the page has
`<meta type="fragment" content="!">` is set.

During the migration to Firebase, this was lost, which resulted in Google
dropping the docs almost completely from the index.

We are using a Firebase cloud function to serve the partials. Since
we cannot access the static hosted files from the function, we have to
deploy them as part of the function directory instead, from which they
can be read.

Related to angular#16432
Related to angular#16417
Narretz added a commit to Narretz/angular.js that referenced this issue Feb 12, 2018
This commit restores serving the plain partials (content) when a docs
page is accessed with ?_escaped_fragment_=.
The Google Ajax Crawler accesses these urls when the page has
`<meta type="fragment" content="!">` is set.

During the migration to Firebase, this was lost, which resulted in Google
dropping the docs almost completely from the index.

We are using a Firebase cloud function to serve the partials. Since
we cannot access the static hosted files from the function, we have to
deploy them as part of the function directory instead, from which they
can be read.

Related to angular#16432
Related to angular#16417
Narretz added a commit to Narretz/angular.js that referenced this issue Feb 12, 2018
This commit restores serving the plain partials (content) when a docs
page is accessed with ?_escaped_fragment_=.
The Google Ajax Crawler accesses these urls when the page has
`<meta type="fragment" content="!">` is set.

During the migration to Firebase, this was lost, which resulted in Google
dropping the docs almost completely from the index.

We are using a Firebase cloud function to serve the partials. Since
we cannot access the static hosted files from the function, we have to
deploy them as part of the function directory instead, from which they
can be read.

Related to angular#16432
Related to angular#16417
Narretz added a commit that referenced this issue Feb 12, 2018
This commit restores serving the plain partials (content) when a docs
page is accessed with ?_escaped_fragment_=.
The Google Ajax Crawler accesses these urls when the page has
`<meta type="fragment" content="!">` is set.

During the migration to Firebase, this was lost, which resulted in Google
dropping the docs almost completely from the index.

We are using a Firebase cloud function to serve the partials. Since
we cannot access the static hosted files from the function, we have to
deploy them as part of the function directory instead, from which they
can be read.

Related to #16432
Related to #16417
Narretz added a commit that referenced this issue Feb 12, 2018
This commit restores serving the plain partials (content) when a docs
page is accessed with ?_escaped_fragment_=.
The Google Ajax Crawler accesses these urls when the page has
`<meta type="fragment" content="!">` is set.

During the migration to Firebase, this was lost, which resulted in Google
dropping the docs almost completely from the index.

We are using a Firebase cloud function to serve the partials. Since
we cannot access the static hosted files from the function, we have to
deploy them as part of the function directory instead, from which they
can be read.

Related to #16432
Related to #16417
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

3 participants