Important Update: Some Community URL Redirects are Under Maintenance. Learn More. .

cancel
Showing results for 
Search instead for 
Did you mean: 
JamesGriffith
Archer Employee
Archer Employee

Every year at RSA Charge or the RSA Archer Summit, datafeeds are a hot topic, both in the track sessions, discussion panels and in casual conversations afterwards. There’s lots to see and do in this feature, but at its heart it is about bringing data from outside Archer, in. And everyone who has used datafeeds knows it would be nice to make them go faster.

 

RSA Archer release 6.4 contains a number of features which put the power to do this into the hands of the person configuring the datafeed. Each of these features is documented in the RSA Archer release 6.4 online help, and in the release notes, but they are worth highlighting with an overview here. Just as there are multiple moving parts within datafeed, so there are multiple places where RSA Archer release 6.4 provides new tuning mechanisms.

 

  • With the introduction of support for XSLT 3.0, data feed transforms can be passed through a faster engine upfront by marking them as XSLT 3.0. This will shave part of the upfront processing time off the feed.
  • Admins have the option of enabling the Hold Calculations checkbox. This step will cause the datafeed to handle any recalculations it triggers differently to normal. Whereas ordinarily a datafeed will add content into its targeted applications, and queue up a recalculation job for any affected fields on related apps, with the Hold Calculations box checked, these jobs are prevented from running while the feed is active. When the datafeed completes, they are combined into the minimum number of jobs needed to recalculate all the records. Because the calculations are not running while the feed is running, the feed goes faster. And because of the smaller number of recalc jobs at the end, the impact on the system is less.
    DelayCalcs.png

  • Datafeeds are designed to save records one at a time. This gives maximum flexibility – I can make record 1 in my source file reference record 2 if I need to. But often times, we don’t need that level of granularity in the mapping. So in RSA Archer release 6.4 there is now a new token available, which will cause the saves to be submitted in batches of the number set in the token (RSA recommends 1000 in most cases). There are certain scenarios (applications with Advanced Workflow enabled for example) where this is disallowed, but for applications that just need to see a lot of raw data pumped into them (e.g. Vulnerability Scan Results), this token could be just what the doctor ordered.
    BCS.png

  • After a lot of data has been loaded into the system, ongoing recalculation work can create stresses. In release 6.4 admins have been given a new and more granular way to manage recalculation schedules across different applications. This way, they can choose to only recalc active records (for example).
    schedule.png

 

These enhancements to data ingest and its further management empower the Archer Admin community to take charge of their data and tune the behaviour of the system around it to gain performance improvements. We’re very excited about this group of features and looking forward to demonstrating them in our upcoming Free Friday Tech Huddles. Please join us to learn about the value they bring to our community.

8 Comments
BodieMinster
Archer Employee
Archer Employee

These are very exciting enhancements!

ScottNess3
Collaborator III

Has there been any report of issues using the BatchContentSave token?  I'm receiving an error that states the token is enabled but will not be used because workflow is enabled.  However, the specific datafeed is running against the Devices application, which has no workflows.  I had a support ticket open, but was never resolved.

JamesGriffith
Archer Employee
Archer Employee

This can happen if Workflow was once enabled on the application in question. I am not aware of a support case being filed for your specific case of it. If you want to follow-up on that, I would expect us to be able to work with you on a script to reset the configuration on the application.

ScottNess3
Collaborator III

James - the support case number is 01157747.  Although the Devices application has never had an enabled workflow, if there is some script to "reset" the configuration thus allowing this token to work, that would be great.

JamesGriffith
Archer Employee
Archer Employee

Thanks. Someone will be following up with you.

JustinLai1
Contributor III

Any chance we could share this script? I have a similar problem enabling the BatchContentSave token.

Anonymous
Not applicable

Reach out to Support, Justin. Scripts are shared on a case-by-case basis.

ScottNess3
Collaborator III

Justin - You'll need to open a case with support.  It required 3 different SQL scripts to clear applications, like Devices, from appearing to have an advanced workflow enabled.  Ended up needing to use the solution on 4 different applications.