Clarify corner case behavior of --device option

[petergrossmann21]

Description

This PR provides an update to the documentation of the --device command line option.

Related Issue

#2767

Motivation and Context

The goal for these changes is to help readers get a more complete understanding of when/how to specify --device. Existing documentation is biased toward the common case

How Has This Been Tested?

I performed local testing on Zero ASIC architecture models to confirm that, when no <auto_layout> tag is present in the architecture file, VPR identifies the smallest of multiple <fixed_layout> tags for which packing is successful for --device auto. Architectures of this type appear to be present in the available suite of files under vtr_flow/arch. I also locally verified with ZA architecture models that VPR can find the <auto_layout> tag when it is listed after <fixed_layout> tags and --device auto is selected. To the best of my ability to determine with manual inspection, an architecture file of this type is not available in the VPR repo. Unfortunately the ZA test case is not currently available for external use.

If we wish to gate this PR by adding such tests to the CI suite I will need a quick walkthrough of how to do this efficiently.

Types of changes

Documentation only

Checklist:

• My change requires a change to the documentation
• I have updated the documentation accordingly
• I have added tests to cover my changes
• All new and existing tests passed

[AlexandreSinger]

LGTM! Thank you so much @petergrossmann21 for looking into this. @vaughnbetz what do you think?

I did verify the claim that "VPR uses the smallest device amongst all fixed_layout specifications into which the design can be packed" when no auto_layout is specified and auto is selected. The code doing this can be found here:

vtr-verilog-to-routing/vpr/src/base/SetupGrid.cpp

Lines 141 to 256 in dc3e5dc

	/**
	* @brief Create a device grid which satisfies the minimum block counts
	*
	* If a set of fixed grid layouts are specified, the smallest satisfying grid is picked
	* If an auto grid layouts are specified, the smallest dynamicly sized grid is picked
	*/
	static DeviceGrid auto_size_device_grid(const std::vector<t_grid_def>& grid_layouts, const std::map<t_logical_block_type_ptr, size_t>& minimum_instance_counts, float maximum_device_utilization) {
	VTR_ASSERT(!grid_layouts.empty());
	DeviceGrid grid;
	auto is_auto_grid_def = [](const t_grid_def& grid_def) {
	return grid_def.grid_type == GridDefType::AUTO;
	};
	auto auto_layout_itr = std::find_if(grid_layouts.begin(), grid_layouts.end(), is_auto_grid_def);
	if (auto_layout_itr != grid_layouts.end()) {
	//Automatic grid layout, find the smallest height/width
	VTR_ASSERT_SAFE_MSG(std::find_if(auto_layout_itr + 1, grid_layouts.end(), is_auto_grid_def) == grid_layouts.end(), "Only one <auto_layout>");
	//Determine maximum device size to try before concluding that the circuit cannot fit on any device
	//Calculate total number of required instances
	//Then multiply by a factor of MAX_SIZE_FACTOR as overhead
	//This is to avoid infinite loop if increasing the grid size never gets you more of the instance
	//type you need and hence never lets you fit the design
	size_t max_size;
	size_t total_minimum_instance_counts = 0;
	for (auto& inst : minimum_instance_counts) {
	size_t count = inst.second;
	total_minimum_instance_counts += count;
	}
	max_size = total_minimum_instance_counts * MAX_SIZE_FACTOR;
	const auto& grid_def = *auto_layout_itr;
	VTR_ASSERT(grid_def.aspect_ratio >= 0.);
	//Initial size is num_layers x 3 x 3, the smallest possible while avoiding
	//start before end location issues with <perimeter> location
	//specifications
	size_t width = 3;
	size_t height = 3;
	std::vector<t_logical_block_type_ptr> limiting_resources;
	size_t grid_size = 0;
	do {
	//Scale opposite dimension to match aspect ratio
	height = vtr::nint(width / grid_def.aspect_ratio);
	#ifdef VERBOSE
	VTR_LOG("Grid size: %zu x %zu (AR: %.2f) \n", width, height, float(width) / height);
	#endif
	//Build the device
	// Don't warn about out-of-range specifications since these can
	// occur (harmlessly) at small device dimensions
	grid = build_device_grid(grid_def, width, height, false, limiting_resources);
	//Check if it satisfies the block counts
	if (grid_satisfies_instance_counts(grid, minimum_instance_counts, maximum_device_utilization)) {
	//Re-build the grid at the final size with out-of-range
	grid = build_device_grid(grid_def, width, height, false, limiting_resources);
	return grid;
	}
	limiting_resources = grid_overused_resources(grid, minimum_instance_counts);
	//Determine grid size
	grid_size = width * height;
	//Increase the grid size
	width++;
	} while (grid_size < max_size);
	//Maximum device size reached
	VPR_FATAL_ERROR(VPR_ERROR_OTHER,
	"Device auto-fit aborted: device size already exceeds required resources count by %d times yet still cannot fit the design. "
	"This may be due to resources that do not grow as the grid size increases (e.g. PLLs in the Titan Stratix IV architecture capture).\n",
	MAX_SIZE_FACTOR);
	} else {
	VTR_ASSERT(auto_layout_itr == grid_layouts.end());
	//Fixed grid layouts, find the smallest of the fixed layouts
	//Sort the grid layouts from smallest to largest
	std::vector<const t_grid_def*> grid_layouts_view;
	grid_layouts_view.reserve(grid_layouts.size());
	for (const auto& layout : grid_layouts) {
	grid_layouts_view.push_back(&layout);
	}
	auto area_cmp = [](const t_grid_def* lhs, const t_grid_def* rhs) {
	VTR_ASSERT(lhs->grid_type == GridDefType::FIXED);
	VTR_ASSERT(rhs->grid_type == GridDefType::FIXED);
	int lhs_area = lhs->width * lhs->height;
	int rhs_area = rhs->width * rhs->height;
	return lhs_area < rhs_area;
	};
	std::stable_sort(grid_layouts_view.begin(), grid_layouts_view.end(), area_cmp);
	std::vector<t_logical_block_type_ptr> limiting_resources;
	//Try all the fixed devices in order from smallest to largest
	for (const auto* grid_def : grid_layouts_view) {
	//Build the grid
	grid = build_device_grid(*grid_def, grid_def->width, grid_def->height, true, limiting_resources);
	if (grid_satisfies_instance_counts(grid, minimum_instance_counts, maximum_device_utilization)) {
	return grid;
	}
	limiting_resources = grid_overused_resources(grid, minimum_instance_counts);
	}
	}
	return grid; //Unreachable
	}

I remembered seeing this code when I was working on the ClusterLegalizer. When the packer creates a cluster, it checks if it needs more clusters than are currently available, if it needs more space it calls a function to grow the device. The function above is what decides should happen if the "auto" keyword is specified. Just thought people would find it interesting.

[vaughnbetz]

Thanks! Looks good; I just suggest a bit of word tweaking (see comments).

[vaughnbetz]

I would delete thse 3 lines as now they are covered by the introduction of --auto above.

[petergrossmann21]

@vaughnbetz I kept the line documenting how VPR does error handling here, but deleted the bullet points below the highlights; I believe this better captures the desired elimination of redundancy. Let me know if additional changes are desired.

[vaughnbetz]

Looks good, thanks!