fix: FrameworkProcessor with ProcessingStep

Update the implementation of FrameworkProcessor to expect
_normalize_args() as the point of interface with Pipelines for
generating final arguments - not get_run_args(). Add source_dir
capability to ProcessingStep for supported processors.

Pass through KMS key configs correctly to Estimator. Refactor towards
extending the pattern of ScriptProcessor/Processor, rather than
trying to decorate/wrap them.

Fix the FrameworkProcessor+Pipelines integration test to 1) actually
use FWProc and 2) actually check the job is able to succeed. Add a
unit test to verify FWProc+Pipelines integration pattern, and expand
scope of other unit tests to better exercise FrameworkProcessor while
SKLearn is still not aligned to it.

Co-authored-by: Verdi March <[email protected]>

Author: athewsey

src/sagemaker/huggingface/processing.py

@@ -121,6 +121,8 @@ def _create_estimator(
             dependencies=dependencies,
             git_config=git_config,
             code_location=self.code_location,
+            output_kms_key=self.output_kms_key,
+            volume_kms_key=self.volume_kms_key,
             enable_network_isolation=False,
             image_uri=self.image_uri,
             role=self.role,