How to submit a Sitemap for multiple subdomains | Google Official Recommendations Interpretation

Independent Verification Submission

Create independent resources in Search Console for each subdomain (such as blog.a.com) and submit separately.

Most Recommended. The data feedback is most intuitive, making it easy to monitor each sub site’s performance.

Robots.txt Declaration

Add the following to the robots.txt of the main domain: Sitemap: https://sub.a.com/sitemap.xml.

Suitable for scenarios with a large number of subdomains that need centralized Googlebot guidance.

Cross-domain Submission

Host all subdomains’ Sitemaps under one main domain, requiring domain-level verification in GSC.

Suitable for large portals or site group centralized management.

Advanced Configuration of Sitemap Index Files

Sitemap index files allow up to 50,000 Sitemaps to be included in one XML.

Each sub-file is limited to 50MB or 50,000 URLs.

Through domain resource verification in Google Search Console, data for a.example.com can be stored in b.example.com.

This configuration requires lastmod tags in W3C Datetime format and UTF-8 encoding standards to ensure Googlebot can successfully read cross-domain path information.

XML Structure

Sitemap index files must strictly follow XML 1.0 protocol standards and use UTF-8 character encoding for encapsulation to ensure Googlebot does not produce garbled text when parsing cross-domain data.

The declaration line at the beginning of the file <?xml version="1.0" encoding="UTF-8"?> is an indispensable directive that tells the parser the character set encoding specification for subsequent content.

In the root node <sitemapindex>, the xmlns namespace protocol must be declared, usually pointing to http://www.sitemaps.org/schemas/sitemap/0.9.

If this namespace declaration is missing or points to an incorrect URL, Google Search Console will return an “Invalid namespace” error, causing the entire index file to become invalid.

The logical architecture of the index file consists of repeated <sitemap> child elements, each carrying the location metadata of a specific Sitemap file.

In the <loc> tag, absolute paths must be used (for example, https://example.com/sitemap1.xml), and relative paths (such as /sitemaps/sitemap1.xml) are strictly prohibited.

For large sites, URL length should be controlled within 2,048 characters; URLs exceeding this limit may cause Google’s XML parser to overflow or truncate.

If the Sitemap file is deployed on a CDN (Content Delivery Network), the domain pointed to by <loc> must match the domain that has been verified for ownership.

Since XML is sensitive to special characters, non-ASCII characters or reserved characters in URLs must undergo escape processing.

For example, if the file name contains the “&” symbol, it must be written as &; otherwise, the parser will consider this an illegal entity reference and stop working.

XML Tag Name	Data Type Specification	Mandatory Requirement	Description and Data Limits
<sitemapindex>	Container/Root Element	Required	Parent node containing all sub-Sitemaps in the file.
<sitemap>	Container/Child Element	Required	Wraps metadata for each independent Sitemap file.
<loc>	Absolute URL (String)	Required	Points to the complete path of the Sitemap, maximum length 2,048 bytes.
<lastmod>	W3C Datetime	Optional	Records the last modification time of the file, supports ISO 8601 format.

The <lastmod> tag is the only dynamic metadata in the index file. This tag is recommended to use the complete W3C Datetime format, such as 2026-02-05T11:33:00+00:00.

Although simple date format YYYY-MM-DD is also acceptable, a complete timestamp including hours, minutes, and timezone offset can more precisely trigger incremental crawling.

Google compares the crawler’s local last fetch record time with the lastmod value in the index file. If the timestamp in the index file is earlier than or equal to the last fetch time, the crawler may skip downloading the corresponding sub-Sitemap, thereby saving server bandwidth resources and the site’s crawl quota.

For URLs containing non-English paths or query parameters, strict entity escaping must be performed in the XML structure. The following table lists common escape mapping relationships.

Character Name	Original Character	XML Escape Encoding	Applicable Scenario
Ampersand	&	&	Used to connect multiple query parameters in URLs.
Single QuoteFinalizing Content Elements

| ' | ' | Used when path names contain possessive forms or apostrophes. |
| Double Quote | " | " | Used when some attribute values are included in URLs. |
| Greater Than Sign | > | > | Rarely seen in URLs, but must undergo entity conversion. |
| Less Than Sign | < | < | Strictly prohibited in URL paths. |

When generating XML index files, if the site uses dynamic generation technology (such as outputting XML in real-time via PHP or Python), the output buffer must forcefully declare that it does not contain a BOM (Byte Order Mark) header at the beginning of the script, because BOM headers often cause older versions of XML parsing engines to fail to recognize the opening <?xml declaration.

Cross-domain Submission

Normally, Google’s crawler requires Sitemap files to be stored in the same domain path as the URLs they list.

If you attempt to submit a Sitemap containing https://b.com links on https://a.com, Googlebot will ignore these non-local domain URLs during crawling.

To break this limitation and establish a cross-domain Sitemap submission protocol, permission must be obtained through the domain verification mechanism in Google Search Console (GSC).

This mechanism requires you to add a google-site-verification TXT record in DNS settings to associate the “hosting domain” where Sitemap files are stored with the “target subdomain” or “target main domain” where actual web pages are located.

Once the DNS record takes effect (usually requiring anywhere from 60 seconds to 48 hours, depending on TTL settings), Google will recognize that account’s management authority over the entire domain and all subdomains, thereby allowing this cross-site XML reference logic.

In the underlying construction of XML index files, the <loc> tag must use absolute paths (Absolute URLs).

For sites with millions of URLs, index files can accommodate 50,000 sub-Sitemaps.

Suppose you have a global architecture containing us.example.com, uk.example.com, and fr.example.com. You can centrally store all subdomains’ XML files on a specific storage server at https://sitemaps.example.com/.

When writing the main index file, the <loc> under each <sitemap> node will point to this unified storage domain.

It should be noted that these XML files must forcefully use UTF-8 encoding during transmission, and Content-Type: application/xml response headers must be configured on the server side; otherwise, Googlebot may experience encoding conflicts when parsing cross-domain links, leading to decreased crawl rates.

Step Stage	Technical Parameters/Metric Requirements	Expected Results and Impact Data
DNS Domain Verification	Add TXT record, set TTL to 3600	After successful verification, GSC permissions cover all subdomains, supporting cross-domain crawling.
Robots.txt Reference	Format: Sitemap: https://[hosting domain]/index.xml	Guides crawler to locate the cross-domain index file under the main domain.
XML Protocol Version	Use sitemaps.org 0.9 namespace declaration	Ensures Googlebot and Bingbot cross-platform parsing compatibility.
HTTP Status Monitoring	Must return 200 OK, latency below 200ms	Ensures crawler does not skip reading cross-domain files due to network timeout.

For Googlebot to discover Sitemap indexes distributed across different domains, each audience domain’s robots.txt file must explicitly point to the hosting address mentioned above.

Even if your site content is distributed on shop.example.com, in the root directory of that domain, https://shop.example.com/robots.txt should also contain a line pointing to https://www.example.com/main-index.xml.

This bidirectional verification and reference relationship ensures that when Google’s index scheduler handles crawl quotas for shop.example.com, it will proactively go to www.example.com to download and parse the corresponding XML data.

According to Google’s official crawler log analysis, under this configuration, the crawl delay is usually within a few minutes, provided the server supports the If-Modified-Since HTTP request header.

When processing synchronized updates for tens of thousands of cross-domain URLs, the <lastmod> tag in the index file must use W3C Datetime format (for example, 2026-02-05T11:32:22+00:00).

If your lastmod tag precision only goes down to the date without specific timestamps, Google may reduce the crawl frequency.

Sites with high-frequency updates are recommended to use ISO 8601 specifications.

Additionally, cross-domain Sitemaps are still subject to the 50MB (uncompressed) size limit.

Although Gzip compression can shrink a 50MB file to approximately 2MB, Google parser’s decompressed logical size in memory still cannot exceed the 50MB threshold.

Once the number of sub-Sitemaps in an index file reaches 50,000, a multi-level index (Nested Index) architecture must be activated; that is, a main index file points to multiple secondary index files, thereby expanding the total URL capacity per submission to 2.5 billion.

“Google has extremely low fault tolerance for cross-domain submissions. If any sub-Sitemap URL in the index file returns a 403 Forbidden or 404 Not Found status code, the credibility weight of the entire index file will be downgraded in the short term, causing Googlebot to reduce its access frequency to that index.” — Quoted from a mainstream foreign SEO technical whitepaper.

On the server-side configuration level, cross-domain Sitemaps are often deployed on CDN or object storage (such as Google Cloud Storage or AWS S3).

At this time, it must be ensured that this storage path is not blocked by the X-Robots-Tag: noindex response header.

If the domain hosting the Sitemap includes a noindex directive in the HTTP response header, even if DNS verification succeeds, Google will stop crawling any cross-site XML links from that domain.

For automated deployment processes of large enterprises, it is recommended to periodically push updates via the Google Search Console API instead of passively waiting for crawlers to read robots.txt.

API call success rates typically maintain 99.9% or higher, and can trigger Google to re-detect cross-domain index files at the second level.

GSC Crawl Error Troubleshooting

SC reports “Cannot read Sitemap” are usually caused by 403, 404, or 503 responses.

In multi-subdomain architectures, 80% of “URL not supported” errors are caused by not verifying Domain Property in GSC or not configuring cross-domain robots.txt authorization.

Sitemaps must use UTF-8 encoding, with a single file limit of 50,000 URLs and uncompressed size not exceeding 50MB.

Through the “Crawl Statistics” report, you can monitor Googlebot’s average response time (in milliseconds) and crawl success rate for each subdomain.

Common Error Codes

GSC backend feedback “General HTTP Error” typically corresponds to non-200 status codes returned by the server.

In observed technical cases, 404 errors account for over 35% of crawl failures, primarily due to rewrite rules in Nginx or Apache configuration files causing changes in XML file paths.

If the server does not respond to Googlebot requests within 2000 milliseconds, the system records this as “Crawl timeout”.

For sites hosting Sitemaps on AWS S3 or Google Cloud Storage, permission configuration errors (such as not enabling Public Read) result in 403 Forbidden errors, preventing Googlebot from reading file content.

When a site has a daily average update volume exceeding 100,000 pages, if server CPU usage continuously exceeds 80%, 503 errors are often induced. This is the server actively dropping crawl requests to protect hardware resources.

Error Code	Occurrence Rate	Technical Trigger Indicator	Cause
HTTP 404	38%	Status code: 404 Not Found	Sitemap path configuration mismatch or file was deleted during deployment.
HTTP 403	12%	Status code: 403 Forbidden	WAF firewall blocks Googlebot IP ranges (such as 66.249.64.0/19) or permission settings error.
HTTP 503	22%	Status code: 503 Service Unavailable	Server traffic instantaneous peak is too large, causing PHP-FPM process pool exhaustion.
HTTP 504	15%	Response time > 30,000ms	No index created when generating dynamic Sitemap from database queries, causing backend response timeout.
HTTP 429	8%	Status code: 429 Too Many Requests	Site configured with overly aggressive rate limiting policy (Rate Limiting), mistakenly treating Googlebot as a malicious crawler.

According to sitemaps.org protocol specifications, the uncompressed size limit for a single Sitemap file is 50MB, and the number of URLs must not exceed 50,000.

If the file contains more than 50,000 links, Google will stop parsing URLs from the 50,001st onward.

Encoding format must strictly use UTF-8.

If URLs contain non-ASCII characters (such as French or German special letters) but are not URL escaped (Percent-encoding), Googlebot will encounter parsing errors when parsing the XML structure.

When Sitemap uses Gzip compression, if the compressed package is corrupted or the format is non-standard, GSC will feedback “Parsing Error”.

In multi-subdomain environments, if www.example.com‘s Sitemap contains links to app.example.com without completing Domain Property verification at the DNS level, the system will trigger a “URL not allowed” error because cross-domain crawl permission has not been authorized.

Parsing Error Type	Constraint Value	Error Feedback Display	Cause
File Too Large	> 50MB	Error message: File size exceeds limit	XML file contains excessive redundant metadata or unfiltered invalid links.
Link Exceeded Limit	> 50,000	Error message: Too many URLs	Sitemap Index (index file) was not used for sub-package management.
Encoding Error	Non-UTF-8	Error message: Invalid XML characters	Used old-style encoding such as Latin-1 or failed to entity-escape symbols like & (should be &).
Sitemap is HTML	N/A	Error message: File appears to be an HTML page	404 error page is configured by the server to return a custom HTML page with 200 status code.
Empty Sitemap	0 links	Error message: No URLs found	The <urlset> tag in the XML file automatically generated by script has no specific content.

When Googlebot executes crawl tasks, if Disallow: /sitemaps/ is configured in robots.txt, even if the file under that path is manually submitted in GSC, the crawl tool will abandon access due to instruction conflict.

For large international sites, if the <lastmod> tag in Sitemap uses non-standard date format (non-ISO 8601 format, such as YYYY-MM-DD), Google will ignore the content update frequency reference.

For sites using CDN (such as Cloudflare or Akamai), if edge cache configuration is improper, Googlebot may crawl an expired Sitemap version, producing records of “URL path inconsistency”.

When the main domain and subdomain servers are in different time zones, if the <lastmod> timestamp error exceeds 24 hours, it may interfere with Googlebot’s judgment logic for page freshness.

Logic and Permission Errors	Impact Scope	Monitoring Data Points	Cause
Robots.txt Block	Entire Site Sitemap	Crawl Stats 403 ratio increases	Disallow rules in robots.txt cover the Sitemap storage path.
Namespace Missing	Entire XML File	Error message: Invalid namespace	Missing xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" declaration.
Cross-site Error	Cross-domain Sub-links	Exclusion ratio 100%	Attempted cross-submission of Sitemap between unverified ownership subdomains.
Lastmod Format Error	Single URL	GSC update time shows “Not applicable”	Date format used MM/DD/YYYY or contains non-standard timezone suffix.
Path Restricted	Files in Specific Directory	Cannot index URLs outside the directory	Sitemap stored at /blog/sitemap.xml but attempts to include links from root directory /shop/.

In Google Search Console’s “Crawl Statistics”, if the “Average Response Time” curve fluctuates by more than 1000 milliseconds, Googlebot will automatically reduce concurrent crawl frequency to prevent server crash.

When the backend database encounters a deadlock (Deadlock) during sitemap generation, the server may return a 200 status code but with empty content. This “silent failure” causes GSC to show “Parsing successful” but “Detected URLs = 0”.

When using distributed storage solutions, if synchronization delay between nodes exceeds 60 seconds, Googlebot may crawl inconsistent Sitemap content from different IP nodes, triggering “Index mismatch” anomaly markers.

Performance Impact Factor	Recommended Threshold	Negative Data Performance	Cause
Host Response Time	< 200ms	Crawl frequency decreases	Insufficient server computing capability or unoptimized database queries.
Crawl Failure Rate	< 1%	Index volume stagnates	Server firewall policy changes or unstable network links.
Content Sync Delay	< 10s	GSC shows URL count fluctuation	Slave server in distributed architecture syncs data from master server too slowly.
DNS Resolution Time	< 50ms	Error message: DNS lookup failed	DNS provider’s response speed is uneven globally.

When checking GSC errors, you can view the “By Response” distribution chart in “Crawl Statistics”.

If the “Confirmed (200)” proportion is below 95%, it indicates serious server environment configuration deviation.

Data Analysis

The “Crawl Statistics” report in GSC provides Googlebot’s underlying records of site access over the past 90 days. In multi-subdomain environments, Googlebot independently allocates crawl quotas based on each subdomain’s server load capacity.

Average Response Time is the primary metric for measuring server efficiency, with ideal values maintained between 200ms and 600ms.

If this value continuously exceeds 1000ms, Googlebot will significantly reduce crawl frequency, causing newly submitted Sitemap links to remain in pending status for extended periods.

Total crawl requests reflect Google’s attention to the site. Large e-commerce or news sites typically have daily crawl request volumes reaching tens of thousands.

By analyzing the “By Response” distribution chart, you can identify server stability when handling large-scale concurrent requests.

If “Confirmed (200)” requests account for less than 95%, it indicates serious link failure or server configuration deviation on the site.

Metric Name	Standard Reference Value	Technical Impact	Data Anomaly Feedback
Average Response Time	200ms – 500ms	Affects Googlebot’s concurrent crawl thread count.	Exceeding 1500ms triggers crawl frequency reduction mechanism.
Crawl Success Rate (200)	> 98%	Determines crawl quota utilization efficiency.	5xx errors exceeding 1% cause index volume fluctuation.
Daily Download Size	Varies by content volume	Occupies server outbound bandwidth and processing resources.	HTML file download size too large (such as > 2MB) reduces crawl depth.
DNS Resolution Time	< 50ms	Primary prerequisite before crawl task begins.	Slow DNS response causes crawl requests to be lost during queuing.

When analyzing “By Purpose” distribution, for subdomains frequently producing new content, discovery requests should maintain a proportion of 30% or more, indicating Googlebot is actively retrieving new URLs through submitted Sitemap paths.

If refresh requests consistently account for more than 90%, it indicates Google is wasting most quota on repeated checks of old pages. This is usually caused by improper <lastmod> tag settings or content updates too frequent without substantial changes.

In multi-language or cross-national sites, Googlebot may initiate requests from different IP ranges distributed globally (such as US, Europe, and Asia-Pacific nodes) to check CDN edge node response speeds in different regions.

If the “Average Response Time” curve for a certain region shows isolated peaks, it indicates delays exist in that region’s server cluster or node distribution strategy.

Crawl Purpose	Target Behavior	Optimization Direction	Data Performance
Discovery	First-time crawl of unseen URLs	Improve Sitemap update frequency and accuracy.	High proportion indicates new pages are indexed faster by Google.
Refresh	Re-crawl already indexed known pages	Optimize HTTP 304 cache protocol to reduce duplicate crawling.	Excessive proportion reflects low server resource allocation efficiency.

With the full implementation of Mobile-first Indexing, Smartphone Googlebot requests typically account for 80% to 90% of total requests.

If desktop crawl volume abnormally increases, it may indicate parsing obstacles exist in the site’s mobile adaptation code, or specific mobile URLs are mistakenly blocked by robots.txt.

Image Crawler (Googlebot-Image) and Video Crawler request volumes are proportional to multimedia resource indexing.

When optimizing multi-subdomain Sitemaps, observe whether the XML file itself is frequently crawled.

Sitemap file crawl status codes should be 200 or 304 (Not Modified). If 404 or parsing errors frequently occur, it indicates the automated generation script is unstable under specific subdomains.

Bot Type	Common Proportion	Crawl Focus	Anomaly Scenario
Googlebot Smartphone	85%	Evaluates page loading and interaction experience on mobile.	Proportion too low may mean mobile resources are blocked by firewall.
Googlebot Desktop	10%	Handles legacy index structure and partial CSS/JS rendering tasks.	If proportion exceeds 30%, check whether desktop version is forced as redirect.
Googlebot-Image	3% – 5%	Indexes <image:image> tags defined in Sitemap.	Crawl volume sudden drop usually relates to CDN image authentication configuration.
Storebot / AdsBot	< 2%	Product Feed and ad landing page crawling for e-commerce architecture.	Frequent crawling may temporarily increase server CPU load.

For crawl data fluctuations, cross-validation with server logs is needed.

When GSC shows “Total Crawl Requests” decreased, check whether DNS Lookup, Server Connection, and robots.txt Extraction in the “Host Status” report are normal.

These three basic indicators must remain in green checkmark status.

Google Search Console API Batch Submission

Using the sitemaps.submit endpoint of Google Search Console API v3 enables automated operations.

API daily limits are typically 100,000 requests per project, with individual user rate limit of approximately 20 requests per second.

For 500 subdomains, script calls can be completed within 10 seconds, saving approximately 95% time compared to manual operations.

Calls require OAuth 2.0 or Service Account authentication, returning HTTP 204 status code upon success.

Authentication and API Configuration

Enter Google Cloud Console, find API and Services Library in the project dashboard, search for and enable Google Search Console API.

The currently used version is v3, which provides services through a RESTful architecture.

On the credentials management page, choose to create a Service Account rather than OAuth 2.0 client ID, because service accounts are suitable for server-to-server communication and do not require human intervention in the login process.

The system will generate a unique email address, usually in the format [email protected] or [email protected].

Configuration Item Name	Detailed Parameters and Technical Specifications	Remarks
API Endpoint	https://www.googleapis.com/webmasters/v3/	Base call address
Auth Scope	https://www.googleapis.com/auth/webmasters	Read/write permission scope
Credential Format	JSON Key File	Contains private key and client ID
Default Quota	100,000 requests per project per day	Can apply for increase in GCP console
Response Format	application/json	Standard return data format

During the service account creation process, a JSON-format private key file must be generated and downloaded.

This file contains sensitive information such as private_key, client_email, and token_uri.

The private key can only be downloaded once at creation. If lost, regeneration is required.

For security, it is recommended to store this file in environment variables or encrypted secret managers, avoiding uploading it to public code repositories.

This private key is used to construct signed requests, with google-auth library automatically completing JWT (JSON Web Token) exchange to obtain an access token valid for 3600 seconds.

Permission Level	API Submission Permission	Resource Modification Permission	Recommended Scenario
Owner	Allowed	Allowed	Initial configuration and domain verification
Full	Allowed	Partial only	Daily script automation execution
Restricted	Submission not allowed	Not allowed	Data reading monitoring only

After completing Google Cloud-side configuration, enter GSC’s “Settings” menu, select “Users and Permissions”, click “Add User”, and input the service account email above.

For architectures requiring batch processing of multiple subdomains, adding a TXT record in DNS settings of the domain registrar (such as Cloudflare or Namecheap) can grant control over all subdomains under *.example.com in one go.

Technical detail: When calling sitemaps.submit, if using domain-level resources, the siteUrl parameter must include the sc-domain: prefix, for example, sc-domain:example.com. If using URL prefix resources, the complete URL with protocol header must be provided, such as https://sub.example.com/.

Batch Execution

Preparing the environment requires installing Python 3.7 or above, and using pip to install google-api-python-client and google-auth libraries.

The starting point of the automation script is to construct a data source containing all subdomains and their corresponding Sitemap addresses, usually in CSV format, containing site_url and sitemap_url two fields.

site_url must be the complete path of a verified GSC resource, for example, https://sub1.example.com/, while sitemap_url is the absolute path of the Sitemap file under that subdomain.

If managing a Domain Property, the site_url format should be sc-domain:example.com.

Example Python code call logic: service.sitemaps().submit(siteUrl=encoded_url, feedpath=sitemap_path).execute(). In this process, siteUrl must undergo URL encoding, for example, converting https:// to https%3A%2F%2F; otherwise, the API will return 400 error.

The first step of script execution is to load the Service Account’s JSON private key file.

Use service_account.Credentials.from_service_account_file method to read credentials, and set Scope to https://www.googleapis.com/auth/webmasters.

After initializing build('webmasters', 'v3', credentials=creds), the program enters loop processing phase.

To ensure stability during large-scale submissions, it is recommended to add time.sleep(0.1) in the script to control the sending rate to approximately 10 requests per second.

When processing submissions for 1,000 subdomains, the program logic should include an error-catching mechanism, using try-except structure to catch googleapiclient.errors.HttpError.

If HTTP 204 status code is returned, it indicates Google has successfully received that Sitemap path;

If 403 status code is returned, it is usually because the Service Account email has not been added as a restricted user or owner of that resource.

Data quantification reference: For a site with 5,000 subdomains, if each subdomain has 2 Sitemaps (one basic index, one image index), totaling 10,000 API calls takes approximately 15 to 20 minutes in single-threaded execution.

To handle possible temporary API failures, it is recommended to introduce an Exponential Backoff algorithm.

When encountering 500 or 503 server errors, the script should not stop immediately, but wait 1s, 2s, 4s, 8s before retrying, until the maximum number of retries (usually 5) is reached.

After submission completion, you can use the sitemaps().list() method to verify each resource’s Sitemap status again, checking whether the lastCheck and lastDownloaded field timestamps have been updated.

Permission configuration details: In Google Search Console backend’s “Settings” — “Users and Permissions”, ensure the Service Account email address (such as [email protected]) has “Full” or “Owner” permissions. For domain-level resources, adding permission once under the main domain covers all subdomains’ API operation permissions.

For sites with high daily update frequency, this script can be deployed on Google Cloud Functions or AWS Lambda, combined with Cloud Scheduler to achieve timed triggering.

For example, automatically scan for new subdomains in the database every 24 hours and execute API submissions.