Home » AWS » AWS CloudSearch – clear domain using PowerShell

AWS CloudSearch – clear domain using PowerShell

Amazon Web Services CloudSearch is a search engine and it’s offered in a SaaS model (as well as Elasticsearch what I’ve already explained here). I’m currently writing an article about it so I’ll skip the way how to set it up and how it works. Unfortunately, CloudSearch isn’t free. It doesn’t cost much if you take a look at everything what you get. For more information on pricing check here.

There’s few issues with CloudSearch. One of them is the fact that there’s no easy way to clear search domain. AWS CLI has no such feature. At StackOverflow, I managed to find scripts for Java, NodeJS, PHP or bash.

Clear search domain with PowerShell

On daily basis for scripting purposes I use PowerShell. That’s why I wanted to have a script which would quickly clear CloudSearch domain. However, the use of PowerShell with AWS requires AWS Tools for Windows PowerShell. You may get it here.

Assuming, you’ve got access to modify and query your domain, next step is to obtain Search Url and Document Url from AWS console and replace it in the script below.

$searchUrl = '[SEARCH_URL]'
$documentUrl = '[DOCUMENT_URL]'
$parser = 'Lucene'
$querySize = 500

function Get-DomainHits()
{
    (Search-CSDDocuments -ServiceUrl $searchUrl -Query "*:*" -QueryParser $parser -Size $querySize).Hits;
}

function Get-TotalDocuments()
{
    (Get-DomainHits).Found
}

function Delete-Documents()
{
    (Get-DomainHits).Hit | ForEach-Object -begin { $batch = '[' } -process { $batch += '{"type":"delete","id":' + $_.id + '},'} -end { $batch = $batch.Remove($batch.Length - 1, 1); $batch += ']' }
       
    Try
    {
        Invoke-WebRequest -Uri $documentUrl -Method POST -Body $batch -ContentType 'application/json'
    }
    Catch
    {
        $_.Exception
        $_.Exception.Message
    }
}

$total = Get-TotalDocuments
while($total -ne 0)
{
    Delete-Documents

    $total = Get-TotalDocuments

    Write-Host 'Documents left:'$total
    # Sleep for 1 second to give CS time to delete documents
    sleep 1
}

 

Then just run the script. Results on the screen (not yet finished):

AWS CloudSearch clear domain

You may find this script on GitHub.

Published by

Mateusz Pustelak

Software Developer with several years of commercial experience, TDD practitioner, DDD/CQRS fan. Currently working for Universal Music Group in London.

34 thoughts on “AWS CloudSearch – clear domain using PowerShell”

  2. Couple issues I’ve had using your powershell script that may help others, that you probably already know.

    The $documentUrl needs needs to include “/2013-01-01/documents/batch” from just the base url found in cloudsearch. Second, if your ID is not solely numerical it needs to be encapsulated with quotes on line 18.

    Reply

  6. Thank your valuable content.we are very thankful to you.one of the recommended blog.which is very useful to new learners and professionals.content is very useful for hadoop learners

    Reply

  7. Great post i must say and thanks for the information. Education is definitely a sticky subject. However, is still among the leading topics of our time. I appreciate your post and look forward to more.

    Reply

  19. Excelr is providing emerging & trending technology training, such as for data science, Machine learning, Artificial Intelligence, AWS, Tableau, Digital Marketing. Excelr is standing as a leader in providing quality training on top demanding technologies in 2019. Excelr`s versatile training is making a huge difference all across the globe. Enable ?business analytics? skills in you, and the trainers who were delivering training on these are industry stalwarts. Get certification on ”
    aws training institute in hyderabad”    and get trained with Excelr.

    Reply

  20. This is a nice article here with some useful tips for those who are not used-to comment that frequently. Thanks for this helpful information I agree with all the points you have given to us. I will follow all of them.

    Reply

Leave a Reply

Your email address will not be published. Required fields are marked *

CAPTCHA

*