Smart Filter Bundles

Description

Smartcrypt Discovery automates the critical task of securing sensitive content throughout the enterprise. It uses a combination of predefined dictionaries and other patterns that you can customize for your unique needs.

Use this page to set up what Discovery should look for, then set Policies to tell Smartcrypt what to do with the discovered data.

Discovery Terms: Patterns and Smart Filter Bundles

Discovery looks for sensitive data by analyzing files and and outgoing email messages to identify common patterns, such as credit card numbers, names of prescription drugs (indicating health information), home addresses and the like.

Smartcrypt provides some predefined patterns to help discover sensitive data throughout the Smartcrypt ecosystem. View some of these patterns on the Distributed Dictionaries page. Define custom patterns by either adding a custom list of search terms, or a custom regular expression (Regex) to help identify what you are looking for in files and Microsoft Outlook email messages.

You can include multiple patterns in one Smart Filter Bundle to identify the data you're seeking to protect in one pass.For example, if you want to protect personally identifiable information, use these existing patterns:

A Smart Filter Bundle is a combination of patterns and threshold quantities. The threshold is the quantity of the pattern where Smartcrypt takes notice. Assign one or more patterns to a Smart Filter Bundle. You can define a Smart Filter Bundle to search for different quantities of each pattern assigned as well.

Continue reading to learn how to create your own customized patterns and bundles.

You'll tell Smartcrypt what to do with files and emails that meet the threshold, called remediation, or action steps taken, when you define when and where to use the Smart Filter Bundle. Smart Filter Bundles are just the rules and regulations for what the Smartcrypt Discovery agents will search for. We still need to tell the agents to do the work. Scanning files on File Servers or Workstations can be set up and deployed through Assignments or Lockers. To enable the Microsoft Outlook Plugin to scan email body and attachments, you can control this behavior through Policies.

Add Discovery

Let's look at how to build that Personally Identifiable Information filter bundle.

  1. Click Add Discovery.
  2. Name the bundle Personally Identifiable Information.
  3. Use the drop-down menu to choose Address US.
  4. The threshold is the quantity of the pattern where Smartcrypt takes notice. Set a threshold of 10 for this pattern.
  5. Click Add to add these patterns: National Insurance Number UK, Social Security Number US and Tax ID US. Give each of them a Threshold of 1.

In this example, if a file is scanned and contains 5 US Addresses, no remediation action will be taken because the quantity is not found. A mailing list with 10 or more US addresses would be flagged for remediation. If a US Social Security Number is found 3 times, a remediation will take place.

In the Add Smart Filter Bundle screen, you may also add Exclusion and Inclusion Filters.

Save the bundle when complete.

Add Redaction

The steps to create a bundle where remediation means redacting text are similar to the Discovery Smart Filter Bundle. You do not have to set a threshold in the Redaction bundle.

  1. Click Add Redaction.
  2. Name the bundle.
  3. Use the drop-down menu to choose one or more Patterns.
  4. (Optional) Add Exclusion and Inclusion Filters.
  5. Save the new bundle.

File Filters

File filters are used to create file and folder bundles to discover based last accessed date, file extension, residing folder name, etc.

ItemDescription
NameUnique name that is referenced from assignment and locker pages
Platform

Windows

  • Ability to exclude hidden files and system files
  • Define relative and absolute dates based on,
    • Created Date - WIN32_FILE_ATTRIBUTE_DATA.ftCreationTime
    • Last Modified Date - WIN32_FILE_ATTRIBUTE_DATA.ftLastWriteTime
    • Last Accessed Date - WIN32_FILE_ATTRIBUTE_DATA.ftLastAccessTime


macOS and Unix

  • Ability to exclude system files
  • Define relative and absolute dates based on,
    • Last Modified Date - stat.st_mtime;
    • Last Accessed Date - stat.st_atime;
Whitelist Extensions & Blacklist Extensions

Define file names and their extensions.

Smartcrypt will process every file in a directory it's defined to discover. With the Whitelist and Blacklist, you can restrict the number and type of files processed in the folder. For example, if you only want to process spreadsheets in this assignment, type *.xls* in the whitelist and leave the blacklist blank. All other files placed in this assignment will remain unprocessed. Files/extensions are separated by semicolons.

Notes

  • Use one at a time or together
  • Blacklist automatically defaults with *.dropbox, desktop.ini, thumbs.db, ~.*,
  • If there is a conflict of the same extension in both rule sets, only the conflicting whitelist item will be processed
  • Both extension lists can be used at the same time in use cases such as
    • Whitelist Extension: "*.doc"
    • Blacklist Extension: "foo*"
    • Result: Discovery will trigger on files with a .doc file type and not pick up files that start with a file name of foo
Whitelist Paths & Blacklist Paths

Define whitelists and blacklists of the residing folder path a file could be discovered in,

  • Use one at a time or together
  • If there is a conflict of the same extension in both rule sets, only the conflicting whitelist item will be processed
  • Both extension lists can be used at the same time in use cases such as
    • Whitelist Paths: "*\Desktop\*"
    • Blacklist Paths: "*\Personal\*"
    • Result: Discovery will only trigger on files that reside in a folder path containing "Desktop" and does not include "Personal"




The steps to create a bundle where remediation means redacting text are similar to the Discovery Smart Filter Bundle. You do not have to set a threshold in the Redaction bundle.

  1. Click Add Redaction.
  2. Name the bundle.
  3. Use the drop-down menu to choose one or more Patterns.
  4. (Optional) Add Exclusion and Inclusion Filters.
  5. Save the new bundle.

Discovery Patterns

Custom Discovery Dictionary

You can identify your own patterns for Discovery to flag. Choose keywords, define a regular expression, or create a Dictionary file to upload.

  1. Click Patterns from the main Discovery page. A list of any existing custom patterns appears.
  2. Click Add Custom Dictionary
  3. Name this dictionary
  4. Type each word to flag in the Keywords field.

    You can also create a list of words and/or phrases to include as a pattern in a spreadsheet or text editor. Save that list as a CSV file.  Each entry should be enclosed in quotation marks (such as “My Entry”). Do not use commas inside the entry. To include this list as a pattern dictionary, click Browse to identify the file, then load the dictionary file into Smartcrypt.

  5. Use the checkboxes to Match Whole Phrase and/or Match Case for the defined keywords
  6. Click Save when all the keywords and phrases are included

Custom Regex (Regular Expressions)

Use regular expressions (regex) for more flexibility in defining a custom discovery pattern.

Adding regular expressions follows the same workflow as adding keywords, with wildcards. Smartcrypt Discovery will flag any text matching the named regex.

Filters

(Optional) Add exclusion and inclusion filters to reduce false positive results from your bundle. These filters are applied after the primary patterns are identified, but before any remediation takes place.

Exclusion filters work like a blacklist; adding a set of digits, words, phrases or regular expression to an exclusion filter will be separated and processed differently from other data in this bundle.

Inclusion filters work like a whitelist; data matching the filter will be treated like any other matching data.

Exclusion Filter

To add an exclusion filter:

  1. Open an existing bundle with the Edit link, or add a new bundle.
  2. Click Add under Exclusions.
  3. Use the drop-down menu to choose a data pattern to exclude.
  4. Continue adding patterns to exclude by clicking Add.

Notes

Exclusion filters must relate to the patterns in the current Smart Filter Bundle. For example, don't exclude US Credit Cards from the Personally Identifiable Information bundle we created earlier.

Use Exclusion filters with patterns, regular expressions or custom dictionaries.

Inclusion Filter

To add an exclusion filter:

  1. Open an existing bundle with the Edit link, or add a new bundle.
  2. Click Add under Inclusions.
  3. Add a keyword, regular expression or dictionary.
  4. Continue adding patterns to include by clicking Add.

Note: Inclusion filters must relate to the patterns in the current Smart Filter Bundle. For example, don't try to identify US Credit Cards for remediation when you are searching for Personally Identifiable Information.

Importing and Exporting Bundle Packs

You can move existing Smart Filter Bundles, including custom bundles, from one instance of Smartcrypt Enterprise Manager to another.

Exporting Discovery Bundle Packs

To export a Smart Filter Bundle:

  1. Click Export Bundle Pack.
  2. Name the file that contains the bundle.
  3. Use the drop-down Filter Bundles menu to select one or more bundle. Ctrl+Click to select multiple bundles.
  4. Click Export. You'll be asked to save the zip archive containing the exported bundle(s).

You'll return to the Export Discovery Bundle Pack screen to create additional bundles. Click Cancel to return to the Discovery page.

Smartcrypt delivers each selected bundle as a CSV file and packages the exported files in a ZIP archive.

Importing Discovery Bundle Packs

To import a Smart Filter Bundle into an instance of Smartcrypt Enterprise Manager:

  1. Click Import Bundle Pack on the Discovery page.
  2. Browse to the file that contains the exported bundle.
  3. Click Import.

Before importing a bundle, Smartcrypt checks for a bundle with the same name exists on this instance. You'll get an error if that happens. All bundles not already on the system will import.

You'll return to the Import Discovery Bundle Pack screen to import additional bundles. Click Cancel to return to the Discovery page.

Discovery Global Settings

You can customize some Discovery engine settings at the bottom of the page:

Discovery Agent Scanning Priority: Should the Discovery Agent scan a system when other applications are running? Set Discovery's priority here. Choose from Low (default), Normal, Below Normal and Idle (only when the device is not being used by another application).

Version Detection Time Frame (Days): The Discovery engine will update versions periodically with different capabilities. When a user's device connects with SEM, that device agent may not support the same Discovery Engine as other devices. If any device registered to this user checks in to SEM within the time frame (in days) set here, SEM will deliver the minimum set of supported capabilities for that user.  If this setting is zero (0), SEM will deliver the maximum version of the Engine. Example: User has an active 15.50.50 client and a 15.60.12 client. If this setting is greater than zero, the SEM will serve this user 15.50-supported Discovery settings. If this setting is zero, SEM will serve 15.60-supported Discovery settings.

Scan Metadata: Check this box to have the Discovery Agent search file metadata (properties) in a Discovery search process.

Scan Alternate Data Streams: Check this box to have the Discovery Agent search alternate data streams (such as zone identifiers) in its search.

Target Content in MS-Office File Formats: Check this box to have the Discovery Agent focus its search on Microsoft Office files (mainly with *.doc* and *.xls* file extensions).

Rescan on Upgrade: Check this box to have the Discovery Agent rescan all files in Assignments and Lockers each time it is upgraded.

Max Redaction Processes: Limit the number of concurrent redaction searches. Default is 2.